LYMBis in the Employee Health IT Industry

LYMB

lymb

How to Keep Your Fonts in Embedded SVG

According to the American Medical Association, physician burnout rate reached almost 63% in 2021, while professional fulfillment scores plummeted to 22%. This seems like the right time for the medical sector to look for a solution. Whether you already have experience with AI or not, implementing natural language processing in healthcare can take some of the load off your employees’ shoulders and streamline your operations. MarketsandMarkets reports that the global natural language processing in healthcare and life sciences market was valued at $2.2 billion in 2022 and is forecast to reach $7.2 billion by 2027. And this trend will continue as more medical organizations turn to to build custom solutions. NLP service companies So, how is NLP used in healthcare, and how to deploy this technology? Keep reading to find out. What is natural language processing in healthcare? Natural language processing (NLP) is a that uses algorithms to extract meaning from unstructured human language presented either in spoken or written format. NLP requires knowledge of computational linguistics and other machine learning skills. branch of AI Natural language processing is gaining popularity in healthcare as it can analyze large quantities of unstructured medical data, such as doctor notes, medical records, clinical trial reports, and even service reviews that patients post on social media. Research shows that about and not utilized to its full potential. NLP can change that by analyzing data and extracting insights to guide doctors and pharmacists to make more informed decisions. 80% of healthcare data is unstructured Here are some of the main NLP techniques that are useful in healthcare settings: . OCR converts printed and handwritten text to a machine-readable format. It can extract text from images and tables and pass it to other NLP algorithms for further processing. If you want to learn more about the topic, check out our recent . Optical character recognition (OCR) guide on optical character recognition . This approach involves assigning semantic labels based on predefined categories. For instance, it allows doctors to label a patient as “at risk of hospitalization” based on certain keywords in their medical notes. Text classification . This is an information extraction model that can pinpoint entities in text. Physicians can use this technique to extract entities, such as “treatment” and “symptoms,” from a heap of medical data. Named entity recognition . This technique can sort information based on common topics. For example, it can group together all doctors’ notes on patients suffering from a particular disorder. Topic modeling . This NLP method can establish semantic relationships between entities. For instance, it can point out that a particular patient was treated at that hospital under the supervision of that doctor. Relationship extraction NLP use cases in healthcare After learning about what healthcare NLP is and how it works, let’s see how it can contribute to the healthcare field. Clinical documentation management Research shows that physicians for every patient they see. This time could have been spent caring for patients. Instead, it’s wasted on the screen and contributes to doctor burnout. spend 16 minutes on EHRs on average Enhancing clinical documentation through speech recognition Natural language processing in healthcare can help physicians work with electronic health records (EHRs). Doctors can use speech-to-text conversion tools with built-in NLP capabilities to transcribe their notes and enter them into the corresponding patent’s EHR fields. Also, medical staff can query the NLP tools to extract relevant data from EHRs. For instance, Nuance’s solution is a cloud-based speech recognition tool that helps medical personnel document patients’ stories in their EHRs. The company claims that deploying their solution will cut time spent on clinical documentation by half. Concord Hospital deployed Dragon Medical One, and more accurate clinical documentation. Dragon Medical One 75% of employees reported Supporting doctors in decision making NLP solutions can also analyze clinical documents and support physicians in real-time decision making. For instance, after analyzing doctor notes, the system can predict hospital bed demands, which gives hospital staff time to prepare and accommodate incoming patients. Medical coding and billing Medical coding means deriving billable information from clinical notes and transferring it into standardized medical codes. Traditionally, a human coder would perform this task. But manual coding is slow and prone to error, meaning that the provider might not be able to claim and receive the full amount in compensations. NLP-powered computer-assisted coding (CAC) tools can derive information from doctor notes and patient EHRs about different therapies and procedures they encountered, and provide the corresponding insurance codes to solidify claims. One example of CAC natural language processing in healthcare is . The company employs over 150 coding experts and remains up to date with the newest regulations. This solution assists coders by aggregating and analyzing patient documentation, offering auto-suggested tags and comprehensive review and approval tools. 3M 360 Encompass System Clinical trial management Studies reveal that around because they can’t recruit enough participants. Medical NLP can speed up recruitment by scanning patients’ medical data in search of eligible trial candidates. The technology can also help people who want to participate in clinical trials. It can transform eligibility criteria into questions, allowing candidates to filter through many trials fast.Headquartered in Canada, Inspirata built a solution that employs natural language processing in healthcare to . Its NLP engine mines clinical documents and transferred unstructured data into structured insights on patient eligibility. 80% of clinical trials are delayed or terminated facilitate patient recruitment Another successful NLP platform, , can not only help recruit patients but also assist in trial design and site selection. It can parse a variety of sources, such as news feeds, patents, medical literature, and similar trial descriptions. Linguamatics NLP Refer to our to discover how other subdisciplines of AI contribute to this field. article on AI in clinical trials Patient sentiment analysis Applying NLP in healthcare helps medical staff aggregate and analyze customer reviews and opinions from different social media platforms. Natural language processing tools can process thousands of reviews to gauge patients’ attitude towards the care they received. Such tools can identify aspects that frustrate patients, segment them based on frequency, and start implementing improvements for the most recurring negative feedback. For instance, a comment such as “ ” indicates a negative experience with the administrative staff. I waited for 30 minutes on the phone to talk to the receptionist Another way of benefiting from healthcare NLP in sentiment analysis is processing survey results. Instead of relying on social media, healthcare facilities can compose targeted surveys, distribute them among patients, and use NLP solutions to automatically review every response. In a recent study, a team of researchers that can scan survey results to determine how patients feel about their healthcare provider. The team designed a survey with open-ended questions, distributed it on vestibular schwannoma patients, and received 534 responses. The algorithm classified the responses into two groups — positive and negative sentiments. This classification had a 90% overlap with the manual classification produced by human experts. built an NLP solution Check out our blog article if you want to learn more about . how sentiment analysis can boost your customer service Drug discovery To find effective candidate drugs, scientists need to identify the biological origins and understand the disease first. This includes reviewing large amounts of medical literature, patient data, and more. Natural language processing in healthcare can quickly sift through all this information, extract what is relevant, and present it to researchers in a compelling format so that they can learn about similar diseases and how they were treated. For instance, Texas-based offers an AI-powered software that can assist in drug discovery by applying NLP to analyze unstructured data from multiple sources, such as gene cards, PubMed publications, proprietary global research data repositories, clinical trial documentation, and more. After absorbing the available disease-related knowledge, the tool builds an ontology of existing drugs that highlights promising research areas to discover new effective drugs. Lymba Additionally, NLP can analyze social media feeds and patients’ medical records to gauge adverse effects of different drugs. Some tools can even correlate a drug’s side effects to the dosage consumed and the frequency of use. For more exciting applications of you can refer to our blog. AI in drug discovery Challenges of NLP implementation in healthcare Applying natural language processing in healthcare has many benefits. But if you decide to enhance your practice with this technology, here are the challenges that you are likely to encounter. Specific language requirements of the healthcare field Doctors have their specific way of writing clinical notes and other medical documents. This style is typically heavy with jargon, acronyms, and abbreviations, and has limited context. To add to the confusion, a third of the clinical abbreviations present in the Unified Medical Language System Metathesaurus . have several meanings each Even words that are spelled out in full can be confusing. For instance, “discharge” can either mean release from a hospital or bodily fluids, depending on the context. And if you want to extend the application of NLP in healthcare beyond your clinic and use it to analyze patient’s feedback on social media, then you need to familiarize it with the specific abbreviations and emoticons that people use to express their emotions instead of words. If you acquire ready-made general-purpose NLP algorithms, you will need to retrain them to operate in healthcare settings. And even better, familiarize them with the vocabulary used by your doctors. Issues related to human language complexity It’s relatively easy to deploy NLP algorithms to spot terms that are explicitly mentioned and unambiguous. But human language is not always that straightforward. Sometimes, algorithms will need to deal with the following: . Natural language processing models in healthcare have to be able to infer information from the input text. For example, if you want to know whether a patient has social support, there might not be a clear mention of this in their medical notes. However, there might be something like “relative at bedside,” that implies the patient has a form of support. Inferences . A basic keyword-driven NLP will perceive the two statements “wife helps patient with medication” and “patient helps wife with medication” as identical, while in reality the semantics in both cases are very different. Semantics beyond keywords . Clinicians often use negation to indicate absence of medical conditions. For example, a doctor might write “a brain malignancy was ruled out” to document the fact that a malignant brain tumor wasn’t found. Medical NLP algorithms can’t mistake this with the presence of the disease. Negation Decide on how advanced your algorithms need to be based on the target business cases. And if you opt for more basic solutions, be aware of their limitations. Bias and explainability Like any other AI technology, NLP in the medical field can acquire during training on unbalanced datasets or when it continues to learn on the job. Gender bias is one of the most prominent types in natural language processing-based tools. For example, with jobs that require high-level education, such as a doctor, while females are connected to less knowledge-intensive occupations, like a nurse. different types of bias GPT-3 tends to associate males Unfortunately, AI bias is not uncommon in healthcare. For example, an algorithm that was supposed to determine the severity of patients’ illness required for black patients just because, historically, more money was spent on white patients’ needs. significantly downplayed the level of care It can be hard to spot bias when the algorithms are “black box” models that don’t explain how they arrive at their conclusions. One solution to this issue is to deploy . With this technology, healthcare natural language processing tools justify their recommendations, allowing you to verify whether they are biased or not. explainable AI Another way to spot bias is using a curated dataset that scientists designed to detect specific types of bias. This solution is not scalable to large applications but works well with limited usage. In addition to the three NLP-specific challenges presented above, you might encounter general obstacles associated with implementing any AI-powered technology, such as: . Many healthcare organizations still rely on outdated legacy systems that aren’t built to work with AI-driven technology and deal with large amounts of data. Integration with legacy systems . Medical facilities struggle to provide sufficient, high-quality data that faithfully represents the target population. Insufficient training data . Hospitals struggle with the responsibility that comes with using AI, such as who has the final say on selecting the right treatment, and who is responsible if the treatment didn’t work. Ethical considerations and moral hazards For a more detailed explanation, check our article on . top AI implementation challenges Step-by-step implementation of natural language processing in healthcare If you want to use NLP in healthcare, you can follow these steps: . Determine which problems you want to solve with NLP and whether you need advanced language processing capabilities, such as negation and inference. Identify potential use cases . Decide if you want to acquire a ready-made NLP tool or build your own customized product. Off-the-shelf solutions are suited for companies with limited funds and flexible internal processes that can be adapted to accommodate an external product. Consider a custom-built solution if you have rigid workflows, are looking for something scalable, and need to integrate the new tool with legacy systems. In the latter case, you will need to look for a trusted to collaborate with. Build/buy an NLP solution AI healthcare solutions company . Even if you opted for a ready-made solution in the previous step, you will still need to retrain it for optimal performance, as mainstream NLP models are not designed to handle the specifics of the healthcare sector. You will need to that is bias-free and representative of your target population. Prepare the training dataset prepare a training dataset . Test the tool for bias, usability, and how it integrates with the rest of the workflow. Train and validate the model . If you want to build and/or adopt AI models in the medical sector, compliance is of utmost importance. Your tools need to comply with the US Food and Drugs Administration (FDA), the General Data Protection Regulations (GDPR), and any other regulatory bodies in the countries of your operations. You can find more about the topic in our recent article on . Make sure your algorithms are compliant healthcare IT standards . If you and hired a tech vendor, they will design and build the NLP tool with the specifics of your system in mind. They will also help you integrate the solution with your legacy systems. Integrate with your existing systems opted for a custom healthcare solution . The effort doesn’t end at deploying natural language processing in healthcare. You need to audit the NLP models to make sure they are still producing the desired results and update them to reflect any changes in the target use cases. Monitor and adjust the algorithms constantly Concluding thoughts NLP has many applications in healthcare. It can help save time on medical coding, facilitate recruiting participants for clinical trials, keep patients happy through sentiment analysis, and much more. To successfully deploy natural language processing in healthcare, dedicate time to finding a trusted who has experience in the medical field. A tech partner will help you with: machine learning vendor Creating and training custom NLP models that: Are free from bias Fit seamlessly with the rest of your system Tailor to your practice’s vocabulary and have the desired degree of complexity Complying with the privacy requirements of the healthcare sector Supporting you in auditing and updating the algorithms when needed Looking to enhance your medical practice with NLP? ! We will help you build a compliant solution tailored to your internal systems and vocabulary. Drop us a line Also published . here

Natural Language Processing in Healthcare: A Path to Adoption

Procesamiento del lenguaje natural en el cuidado de la salud: un camino hacia la adopción

Traitement du langage naturel dans les soins de santé : une voie vers l'adoption

हेल्थकेयर में नेचुरल लैंग्वेज प्रोसेसिंग: ए पाथ टू एडॉप्शन

医療における自然言語処理: 導入への道

Processamento de linguagem natural na área da saúde: um caminho para a adoção

Xử lý ngôn ngữ tự nhiên trong chăm sóc sức khỏe: Con đường dẫn đến việc áp dụng

医疗保健中的自然语言处理：采用的途径

It's the end of the world as we know it, humanity is programmable and our operating system is getting a reboot. The question is when the system is back online what will be in its place? With the recent outbreak of global pandemic COVID-19, the world is being pushed further into digital space, where nation-states, corporations, and a New Commons Movement are engineering economic and social systems in a race to create the new matrices in which we will exist. These next-generation frameworks will forever change the way humans interact, transact, live, and evolve. A Brave New World...or Is It? Over the last decade, blockchain technology, cryptocurrencies, smart contracts, and token economics have been an experiment into the possibility of new political-economic systems that are increasingly decentralized. However, this is just the most recent of many attempts throughout history to democratize the political-economic landscape, and although this time new technologies give us a greater chance at success, there are huge questions about the possibility that people in this world will truly ever be free. These questions teeter without end on the precipice of evolutionary advancement. Since the dawn of man, there has been a battle for power and resources spurred by the hegemonic, colonialist, violent, and extractive tendencies of human nature under the reactionary nature of the reptilian part of our brains. This is best illustrated in the iconic Dawn of Man scene from 2001 Space Odyssey by Stanley Kubrick. Operating from a scarcity mindset, these behaviors have dominated and defined the vast majority of our recorded history. The few exceptions, like a Phoenix rising from the ashes, have seen great human advancements as a direct result of various forms of collapse i.e The Enlightenment after the Bubonic Plague. Whether economic, political, or religious in context, the narratives and myth of pursuit of liberation have common threads, and has been seen many times throughout history. When humans come to face the realities of their own mortality and look extinction in the face, there is a moment of pause - continue the status quo, or evolve. In step with the , which has often been attributed as a factor in by researchers, our generation now is experiencing, in its new awareness and response to a global pandemic, the possibilities to reboot and re-evolve into the next phase of humankind. Psychedelics Renaissance mammalian evolution There is an opportunity once again in collapse and the detritus of a Pandora's box of unforeseen technological repercussions, to reflect on and re-imagine what this next iteration of humanity looks like. Will the reptilian brain prevail ushering in a new Dark Age? Or will the growing Lymbic threads propel us closer to a Regenaissance - a Utopian reality, where an operating system based on abundance leverages technology and resources to provide, at minimum, safety, shelter, basic needs and well being to all life on this planet? When the pandemic dust settles, an indication of the results of the political-economical-evolutionary space race will be unveiled. (Or perhaps we can point to certain indicators now of what that could look like.) But before we delve into the effects, we should look at the causes and impetus for this massive shift in paradigms, or realities, that we are experiencing now. Liberty or Extinction By Jessica Zartler under the DWTFYW Public Licence The system is broken. It has been broken for some time in this iteration of humanity and although we have all known at some level, our malaise has reached a point in space-time where it can no longer be tolerated. The global warning signs of impending frog boil have our instincts screaming. The colonialist consequences of the Reptilian Brain Era, fueled the engine of exponential extraction and destruction during the Industrial Revolution, which happened too quickly for the developing universal consciousness - Lymbic threads - to catch up. However, with the growth of globalization and the memetic nature of shared communication - the new Alexandrian library (the internet) and other technologies creating the hive mind - the speed at which we are destroying ourselves, is being mirrored by the speed at which we are becoming aware of ourselves and the destruction. New feedback mechanisms and the increased velocity of these growing mycelial-like systems of connection and coordination map and mirror a meta-organism which we can now see and measure. (Similar to the expansion in perception humanity experienced when we saw the first pictures of the Earth from space). Now as we are approaching a point known as the singularity, or a kind of karmic pause for those a little more spiritually oriented, we have a choice at this present moment, in relative space-time, that is not fixed or pegged to a yet-future. The actions we take now, in this kind of wormhole or portal, are accelerated and exponential with this dimensional shift in perception from tribal orientation ( ) to meta-organism, from which future fractals will be spun. We have more awareness and visibility into the future than ever before, into the effects of our choices via predictive modeling technologies and recent developments of a new space race. see Marshall McLuhan ’s The Gutenberg Galaxy The Cyber-physical Space Race This transitional point in space-time is marked by a new race, but unlike its predecessors, this race is not for external markers of territory or technological supremacy, it is for control over system and incentive design, and thereby the inner recesses by which we can program human behavior and condition responses via economic and social stimuli. When tied with machine learning and algorithmic automation, the protocol and entities that win, could dictate thousands of years of further enslavement or, could map a road to seemingly elusive freedom and abundance (if we make it through the impending environmental calamities). Arguably, the most prominent starting line of the race was when Satoshi Nakamoto succeeded in seeding a new era of programming. While industrial, war-era scarcity-coded systems and bloated bubbles were subsidized by governments’ hyper-inflationary policies and , a new intellectual oligarchy capitalized on Satoshi’s vision, realizing that those with the understanding and power to create new economic dynamics out of that ether could create the rules in the present-future, by manipulating the possibilities and rules of a new operating system based on liquid token economies. the further creation of debt With machine learning, economic manipulation was now decentralized much, in the same way, the printing press decentralized publishing. This time, it is the printing press of money that is fracturing paradigms and unveiling a new wave of unforeseen sociological-evolutionary effects. This technology, which has the exponential ability to move worlds, was unleashed without a tried and tested democratized governance layer, a with enough understanding to act as a watchdog, and other stop-gap measures which could act as check and balance to this new power. fourth estate Operating outside of, or propped up by, nation-states, corporate entities and members of the new intellectual oligarchy have been vying for power. The ICO craze era between 2016-2018 was a bit rushed to claim and swallow large amounts of value in the now “legacy” operating system, to fuel rockets of virtual and corporate-run hegemonies hoping to own the next global economic system, or stake claim to as much of the global network and control over human programming as possible, by literally staking on blockchains in . rigged systems The networks were idealized by Satoshi to be resilient to centralization however, much like with internet 1.0 and the dot com era, not even Satoshi and stewards of decentralized models could foresee how this technology created for liberation once again, could be exploited for deeper enslavement. With Web 2.0, a similar race occurred, with hopes that democratic processes would run free. But in the aftershock of that technological wave, we realized that the tools were being used to weaponize and hijack our bodies, using our very own biological systems, to keep us hooked to the dopamine drips of social media, while harvesting our data and psychometric profiles to extract and shift more and more wealth and resources into these virtual corporate states, and control political outcomes, and by proxy, us ( ). see The Great Hack documentary Now, with Web 3 the stakes are higher as we move into a post-nation-state, cyber-physical governance era. And here, the race to maintain any shred of democracy relies on the underdogs - the success of open source technology, token engineering for public goods, and the emergence of the New Commons Movement. New Commons Mycelial Networks vs. Goliath Screenshot from the documentary Fantastic Fungi showing an illustration of underground mycelial network “Life finds a way”, as so eloquently stated by Jeff Goldblum’s character Dr. Ian Malcolm in Jurassic Park. What many may not know is that Malcolm was a mathematician specializing in chaos theory (apropos for these chaotic times) however, simultaneously emergent with chaos is order; the paradox is the language of universal creation and drawing patterns, whether placebo or not, can create powerful narratives drawing that order into existence. The order is growing like mushrooms - lots of them. Beneath and fueled by the technical, economic, and cultural detritus of the first two iterations of the world wide web (networked neural hive) is an ancient-future intelligence system that nature is regenerating. Enter the protagonist - the New movement. Commons While China’s monolithic project and dystopian nightmare grabs headlines alongside the monopolistic flag of Libra, there is a literal movement taking root and propagating by its own organic, emergent, memetic virality and through the means of ; nature is as nature does - there is balance in The Force. Digital Silk Road Social Credit System Grassroots Economics collective intelligence Rather than pillars, hierarchies, linear engineering and proprietary defensive systems that have developed directives of self-preservation over efficacy, this new movement’s inclusive models of radical open source collaboration, shared stewardship, non-linear routes to emergent solutions and the nascent processes by which organizations form and dissolve for specific purposes, are allowing for rapid growth and meta-programming, seeding a parallel operating system to these monoliths. These decentralized collaborations operate more like organisms, and mimic the natural amalgamation and dispersal patterns found in nature. These processes are not new, but a regenesis. Indigenous cultures around the world, during the height of their civilizations, flourished for thousands of years under sophisticated ecological-economic systems grounded in biomimicry. Concepts of bioregional, surplus economies, and peaceful commerce, which provided institutionalized generosity, allowed for peaceful trade and relations between First Nations, and rooted resource management, governance, and culture in abundance and reciprocity. ( ) see “Resilience Reciprocity and Ecological Economics” by Dr. Ronald Trosper While monolithic monopolies plant economic monocultures for manipulation and homogenized social systems, these New Commons mycelial networks are planting economic permacultures and engineering for enlightenment. For every monolith, there are thousands of new collaborations like The , , , , , , , , , , , , and more... Commons Stack BlockScience Commons Engine Regen Network Earth Ledger Metagov Aragon P2P Foundation Verus Holochain Radical Exchange Open Collective Seeds These bottom-up neural webs are taking root and anchoring the much-needed distribution and circulation mechanisms to revive dying parts of the meta-organism, in economic, environmental, and human resources. More tangibly, they are providing the counter systems, , and checks and balances to powerful new technologies in Web3 and the Deep Web. governance layers Although these New Commons movements and their open-source systems are under-resourced, underrepresented, and undervalued in the face of the internet Goliaths in our existing systems, the energy they do have is arguably much more powerful. It is the energy that turns tides, lifts mountains, and transforms horizons. If evolution is any indicator, the mushrooms have a fighting chance. Darwin is Rolling in His Grave I just wanted an excuse to use this Darwin meme - and other hopes for the humanity we all wish to see. If you haven’t already, you should search “Darwin Memes” If “survival of the fittest” isn’t the most overused and oversimplified cliche used to justify inequality, I don’t know what is. There has been much ado about Darwin’s words and using them to justify socio-economic policy through a very limited lens. What you don’t hear as much, is that his holistic view of nature included "dependence of one being on another". Collective intelligence research supports that when it comes to problem-solving, open-source collaboration is superior in addressing systemic challenges. This is reflected in tech as well. According to (1998) and JC Herz (2005), intelligence will eventually generate superior outcomes to knowledge generated by proprietary software developed within corporations ( 2008). Investors and markets are already starting to reflect and . Eric S. Raymond open-source Flew predict the impending rise of open source and shift funds To be fair, however, collaboration extremists have also gone a little wild in interpreting Darwin’s words. Competition and collaboration both have their places, but what we should really be looking at in the context of complex systems research and building human ecosystems informed by Mother Nature’s law is an organism's “saturation point”, and the research of evolutionary biologist and futurist, . She is a living systems pioneer who points to the collaboration threshold as the focal point, rather than debating Darwin’s research. Elisabet Sahtouris If there is something else we have learned about evolution, it is that diversity is key for survival. Award-winning geneticist and broadcaster Dr. David Suzuki has written extensively on the topic and say now, more than ever, we must heed nature's lessons. When building human systems amidst the rising tides, the re-localization of operating systems creates fractals of resilience and adapts much more quickly to meet challenges; economic polymorphism is a necessity. The key to ensuring diversity and meeting at the collaboration threshold is our ability to coordinate. Commons systems, growing on the trusses of decentralized networks, are the best hope for humanity. I’m rooting for the underdogs, and so should you. The Dawn of the Regenaissance Nature is the biggest paradox of all. Resilient, powerful, violent, crushing, and tempered over millions of years; it is also tender, vulnerable, pregnable, withered, and endangered. To survive the extinction feedback loop we’ve initiated, we must re-align technology with nature herself, who holds the keyholes to unlock the solutions existing within her codes. As a species, it is time to mature and acknowledge our humble place in the cosmos and our interdependence on each other. By drawing on wisdom from our past and weaving ancient-future technologies, we can harness and scale mechanisms for diversity and collective intelligence by engineering our digital space systems in the patterns of nature, channeling our socio-political-economic system flows into positive feedback loops for the regeneration of our communities and our planet. By learning from the past we can become better futurists, and even better - more conscious present-ists. Narratives are powerful and so are we. Together, let us write the pages of the Regenaissance and the greatest comeback story the world has ever seen. Let us loose the evolutionary shackles of our own creation, and rise again out of the ashes, never to return. Thank you for reading! And a special thank you to Jeff Emmett for his feedback, edits and support, and Camille Corti & Bryan Doreian for their encouragement in the creative process. I would also like to acknowledge the noosphere and  stand in gratitude for all of the researchers who have contributed to the expression of these collective ideas.

Reboot: A New Operating System for Humanity 

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References VIII. CONCLUSION Performance is an aspect that should always be taken into account when creating Lambda functions, especially if you plan to combine them with Java. Once the performance improvements offered by these approaches for using Java with AWS Lambda have been quantified. It is evident that Java can be used competitively with AWS Lambda without the need to resort to runtimes like Node.js or Python (which usually report better performance). Having tested various combinations of proposed strategies and demonstrated that the performance is similar to what can be achieved with the use of GraalVM, we recommend avoiding its usage. We consider it essential to profile the Java functions to select the optimal memory configuration and using AWS SDK v2. From there, optionally, the next step is to the rest of the techniques to achieve one of the proposed combinations. Additionally, following the general best practices such as initializing connections outside the handler or reduce the bundle size, among others, also contribute to optimizing Java functions. The optimization strategies we propose are not the only ones available, there are many other techniques that can similarly yield excellent results. It is important to highlight that the process of optimizing a function is linear in terms of difficulty. In other words, it’s relatively easy to obtain a significant improvement, but if you want to squeeze out the maximum performance, the process requires trial and error and greater effort. It’s also important to highlight that the percentage of improvement offered by each approach depends on the specific use case in which they are applied. This percentage can vary, either increasing or decreasing. REFERENCES [1] H. Puripunpinyo and M. Samadzadeh, “Effect of optimizing java deployment artifacts on aws lambda,” in 2017 IEEE Conference on Computer Communications Workshops (INFOCOMWKSHPS), 2017, pp. 438–443. DOI: 10.1109/INFCOMW.2017.8116416. [2] D. Bardsley, L. Ryan, and J. Howard, “Serverless performance and optimization strategies,” in 2018 IEEE International Conference on Smart Cloud (SmartCloud), 2018, pp. 19–26. DOI: 10.1109/SmartCloud.2018.00012. [3] D. Jackson and G. Clynch, “An investigation of the impact of language runtime on the performance and cost of serverless functions,” in 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), 2018, pp. 154–160. DOI: 10.1109/UCC-Companion.2018.00050. [4] O. Hosseini Mehrshad Sahragard, “Aws lambda language performance,” 2019. [Online]. Available: https://gupea.ub.gu.se/handle/2077/62454. [5] A. L. Dowd, “Performance of java in function-as-a-service computing,” University of Texas Libraries, 2020. [Online]. Available: https://repositories.lib.utexas.edu/handle/2152/84357. [6] S. O’Toole, “Aws lambda java tutorial: Reducing cold starts,” 2020. [Online]. Available: https://www.capitalone.com/tech/cloud/aws-lambda-java-tutorial-reduce-cold-starts/v=1694091771423. [7] D. Quaresma, D. Fireman, and T. E. Pereira, “Controlling garbage collection and request admission to improve performance of faas applications,” in 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 2020, pp. 175–182. DOI: 10.1109/SBAC-PAD49847.2020.00033. [8] M. Sipek, D. Muharemagi´c, B. Mihaljevi´c, and ˇ A. Radovan, “Enhancing performance of cloudbased software applications with graalvm and quarkus,” in 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO), 2020, pp. 1746–1751. DOI: 10.23919/MIPRO48935.2020.9245290. [9] Q. I. S. Hassan B. Hassan Saman A. Barakat, “Survey on serverless computing,” Journal of Cloud Computing, 2021. [10] C. D. Ron Mosquera Bryan Alexander y Villegas Armijos, “Mejora del rendimiento de aplicaciones serverless basadas en la nube con graalvm y quarkus,” 2022. [11] StackOverflow, “Developer survey 2022,” 2022. [Online]. Available: https://survey.stackoverflow.co/2022/. [12] Q. Wu and L. K. John, “Performance of java in function-as-a-service computing,” in 2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC), 2022, pp. 261– 266. DOI: 10.1109/UCC56403.2022.00047. [13] Datdog, “The state of serverless,” datadog, 2023. [Online]. Available: https://www.datadoghq.com/state-of-serverless/. [14] AWS, “Best practices to work with aws lambda functions,” [Online]. Available: https://docs.aws.amazon.com/es es/lambda/latest/dg/best-practices.html. [15] AWS, “Developer guide - aws sdk for java 2.x,” [Online]. Available: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/home.html. [16] AWS, “Improving startup performance with lambda snapstart,” [Online]. Available: https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html. [17] AWS, “Java lambda functions customization settings,” [Online]. Available: https://docs.aws.amazon.com/lambda/latest/dg/java-customization.html. [18] AWS, “Lambda instruction set architectures (arm/x86),” [Online]. Available: https://docs.aws.amazon.com/es es/lambda/latest/dg/foundation-arch.html. [19] AWS, “Profiling functions with aws lambda power tuning,” [Online]. Available: https://docs.aws.amazon.com/lambda/latest/operatorguide/p [20] G. org, “Graalvm overview,” [Online]. Available: https://www.graalvm.org/latest/docs/introduction/. [21] A. Stefano Buliani, “Best practices for aws lambda and java,” [Online]. Available: https://d1.awsstatic.com/events/reinvent/2019/REPEAT 1 B

Performance Best Practices Using Java and AWS Lambda: Conclusion and References

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References VII. RELATED WORK As far as we know, and up to this date, there aren’t too many articles seeking to enhance the Java approach with AWS Lambda in the same manner as we do. Nonetheless, there are some similar studies. In [21] Buliani addresses part of the performance issue and proposes and applies a series of techniques incrementally, analyzing the overall improvement step by step. [12] investigates the performance impact of cold start and containerization for Java-based FaaS functions and compares it to an alternative using Python. [2] takes a similar approach, suggesting a variety of ways to enhance the performance of AWS Lambda but without integrating it with Java. In [5] Dowd investigates the overhead and cold start for functions written in Java, attempting to determine the extent to which the findings depend on the programming language. There are also other studies that have a certain relationship with ours, either because they explore some of the techniques we propose or because they are related with the performance of AWS Lambda. [3], [4] analyze the performance of the various managed runtimes provided by AWS for executing code in different languages on Lambda functions without optimizing it. In [1] Hussachai et al demonstrates how optimizing the Java artifact results in superior performance during the execution of the Lambda function. In [7] evaluate the use of a garbage collector control strategy concerning performance improvement, that is an interesting strategy that we didn’t take in considerationin our study. Related with GraalVM, [8], [10] incorporate this tool with the aim of leveraging the advantage it offers in terms of performance.

Performance Best Practices Using Java and AWS Lambda: Related Work

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References VI. DISCUSSION In Figure 4 and Figure 5, you can see a summary of the cold start improvement and warm Lambdas, respectively. Next, we will analyze each approach and the possibilities they offer. Starting with an appropriate configuration for each function, in our opinion, is the minimum optimization that any function should have, both to improve performance and to balance cost. In addition to offering a significant improvement, it is perfectly compatible with all the other techniques mentioned in this article. Continuing with the use of Snapstart, this technique focuses on mitigating cold starts. In our experience, this technique could not produce as significant an improvement as to justify the incompatibilities it generates, including the lack of support for the arm64 architecture or custom runtimes, along with other important features such as the capability to use EFS or to attach the lambda to a VPC. The improvement offered by using the Arm64 architecture seems to depend on the specific use case in which it is employed. For example, it can be particularly beneficial for compute-intensive applications such as high-performance computing. This could explain why the improvement in our case is not too noticeable. Nevertheless, its use is highly recommended as it can be combined with other approaches discussed, and its adoption is on the rise. The major drawback is the incompatibility with some third-party technologies or dependencies related to this architecture. The use of AWS’s SDK v2 for Java applications is also practically mandatory, as long as it supports all the services and the majority of the dependencies integrated into the function. Thanks to its improvements and modular approach, this second version of the SDK significantly increases performance. Moreover, it is perfectly mixable with the rest of the techniques without adding notable limitations. Combined with version 1, if any unsupported features are needed, they can be utilized as required. As expected, leveraging the benefits of ahead-of-time compilation generates a significant performance boost both in cold starts and when functions are already warm. However, the main drawback is the complexity that this approach adds. It’s common to encounter runtime issues that may require the developer to deal with low-level aspects, and any modifications can become challenging. In principle, it is perfectly combinable with the rest of the techniques that support custom runtimes. The possibilities offered by the JAVA TOOL OPTIONS environment variable are quite extensive, allowing for significant Java VM configuration. In our case, we only tested tiered compilation, but there are other possibilities such as configuring garbage collector behavior [7]. In any case, it’s worth trying to use this variable and integrating it into Java functions whenever the JVM is used because it doesn’t impose any limitations, and the improvement is significant. It’s also important to highlight that investing efforts in refactoring the function’s code and applying the details mentioned in subsection IV-G can achieve an even greater performance improvement than the techniques discussed. This work is entirely dependent on the use case and requires trial and error as well as skill. Regarding the simultaneous application of techniques, for our combinations, we recommend the use of either of the two, so you should choose the one that provides the most benefit, either in terms of performance or greater adaptability to the specific use case.

Performance Best Practices Using Java and AWS Lambda: Discussion

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References V. COMBINATIONS After defining the proposed techniques and quantifying the effects of their application, it becomes essential to attempt to combine them with the intention of detecting synergies, incompatibilities, and understanding the improvements that can be achieved by applying several of these. With this intention, and due to the unavailability of Snapstart in combination with the Arm64 architecture, the following combinations are proposed: • Best memory configuration + SDK v2 + JAVA TOOL OPTIONS environment variable + Snapstart • Best memory configuration + SDK v2 + JAVA TOOL OPTIONS environment variable + Arm64 architecture Two new prototypes have been created, optimized with the previous optimization combinations, and their performance has been measured. The results are compared with each other and independently with the GraalVM prototype. In Figure 3, you can observe the percentage of improvement compared to the performance offered by the initial system, both for the cold start of functions and for the overall performance.

Performance Best Practices Using Java and AWS Lambda: Combinations

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References IV. BEST PRACTICES AND TECHNIQUES Below are the proposed techniques and approaches to reduce the cold start of Java functions and improve their overall performance. Along with their measurements in terms of performance tests. The proposed practices will be compared in the VI.Discussion section. A. Better suited configuration Increasing the allocated memory for a function is probably the most well-known way to reduce cold starts and also the simplest. The allocation of memory to a function is related to the amount of CPU assigned to that function. A greater amount of CPU generally makes the initialization of the function faster. However, it’s not always the case that more memory equals better performance. It’s also necessary to consider that higher memory allocation will result in higher costs, but if the computing time is reduced due to increased performance, the overall cost will decrease as well. To properly balance the memory configuration of a Lambda function, the Lambda Power Tuner solution [19] can be of great help to profile the function, displaying graphs such as the one shown in Figure 2 for Catalog lambda of our solution. In our case, chose the right memory value for the functions configuration suppose an improvement of about 70% in terms of cold start going from an average of 16307 ms to 4867 ms. In addition to a noteworthy 46% when the functions are warmed passing from an average of 512 ms to 216 ms as can be seen in Table II. For the rest of the versions that do not have an appropriate memory configuration, the default value of 512 MB is used. B. Snapstart Snapstart [16] is a feature offered by AWS to mitigate the cold start of Java functions. It involves initializing the function at the moment a version of it is created. Lambda takes a snapshot of the initialized lambda’s runtime environment, encrypts and caches it. When initializing new functions, it does so from the snapshot rather than starting from scratch, thus improving performance. A notable drawback of this feature is that it does not support many other features offered by AWS Lambda, such as custom runtimes, the arm64 architecture, Amazon EFS, and other managed runtimes that are not Java 11 or Java 17. In this case, enable Snapstart on our functions allow an enhancement of 16% at the cold start (going from an average of 16307 ms to 13736 ms) and an improvement of 21% attending requests when the functions are warmed making a difference of over 100 ms in each request answered (Table III). C. Arm64 architecture The instruction set architecture within AWS Lambda, either x86 64 or arm64, determines the processor that the function will use. AWS’s official documentation suggests that the arm64 architecture [18] with its Graviton processor can offer significantly better performance than the x86 64 architecture. Regarding our solution, the impact of using arm64 on the latencies was positive, getting over a 14% of improvement on cold starts and a 13% when the lambdas are warmed. Reducing the average latency by over 2000 ms and approximately 70 ms, respectively. The warmed lambda functions data can be seen in Table IV. D. AWS SDK v2 for Java The AWS SDK for Java 2.x [15] is the second version of the SDK provided by AWS for Java. This version is designed with a modular approach, allowing you to include only the service-specific JARs you need, reducing the size of your artifacts. That’s a key point of reducing the cold start, loading less number of classes when initialize the function. Furthermore, it boasts several other performance improvements such as asynchronous nature, connection management, concurrency support, resource management, and so forth. Due to that, in terms of performance its use is highly recommended. For us, migrate the lambda functions from sdk v1 to sdk v2 suppose an enhancement of about 40% in both cold start and warmed functions. On first going from 16307 ms to 9926 ms and on second one passing from 512 ms to 298 ms of latency as is visible in Table V. E. GraalVM GraalVM [20] is a platform that enables ahead-of-time compilation of Java code, leveraging the Native Image tool to produce a standalone binary. As a result, this binary offers significantly better performance compared to traditional Java code that runs using the JVM and is compiled at runtime. The binary is produced for the OS of the environment the code is built. It’s important to note that AWS Lambda is only compatible with GraalVM through the use of custom runtimes. Other disadvantage is that the ahead-of-time compilation sometimes would be tricky to make it work properly adding even the necessity of dealing with low level issues caused by AOT compilation. For us, create a native image of our code and uses it generates a great improvement of 83% on cold start, going form 16307 ms to 2800 ms. When the lambdas are warmed, it goes from and average of 512 ms to 231 ms which means a 55% of enhancement, visible in Table VI. F. Environment variable: JAVA TOOL OPTIONS AWS Lambda provides some customization options [17] for the managed Java runtime, which can enhance overall function performance. This is achieved through the use of the JAVA TOOL OPTIONS environment variable, which allows access to JVM features such as tiered compilation or garbage collector behavior among others. In this case we use only tiered compilation, fixing the level to 1, enabling the C1 compiler which means optimized code for fast start-up time. The exact value for the mentioned key is ”-XX:+TieredCompilation - XX:TieredStopAtLevel=1”. Java 17 managed runtime is default set to level 1 for AWS Lambda, while in Java 11, you need to set it manually. Level two can be used for an overall better performance, not only focus on the start time. In our test system, enabling that option translates to a 40% improvement in cold start times and a 31% improvement in response times for the system with warm Lambdas (Table VII). G. Others In addition to the previous techniques and approaches we propose and have tested, there are many other best practices [6], [14], [21]. Whether they are more dependent on the use case or more general, below, we mention some of them: • Reduce the size of the bundle to enhance the initialization phase. As less classes to load as fast the function will initialize. • Initialize the connections of third party out of the handler and same the static resources on /tmp to reuse it from one execution to another. • A framework, if needed, should be as lighter as possible. Due to that we suggest replacing spring with other smaller frameworks like Vert.x • Try to avoid reflection, that feature makes some JVM optimizations impossible to be done as it works with dynamic types. • Initialize all necessary dependencies and classes during initialization time. These approaches, combined with some of the previous ones we have tested, for sure will greatly reduce the impact of cold starts and increase the performance of Java functions by a really significant percentage.

Performance Best Practices Using Java and AWS Lambda: Best Practices and Techniques

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References II. INITIAL APPLICATION A common use case for AWS Lambda is to integrate with DynamoDB, allowing functions to perform operations on data stored in DynamoDB. Following this approach, an initial solution has been built to measure its initial performance and demonstrate the performance improvements generated as a result of the proposed best practices. The mentioned system consists of a simple e-commerce solution, compound of three services with functionality which translates in one function for each service. All the services are self-contained and individual, offering an API through of which the services send messages (using AWS API Gateway). The three services are the following: • Catalog service: It manages the operations related with the products. • Customer service: It manages everything related with customers. • Order service: It involves all in relation to orders. The scheme of the system will be better explained in the Figure 1. The performance of this initial solution can be seen in Table I with an average latency of 512 ms when lambdas are warmed and over 16307 ms on cold starts. It is important to highlight that the test lambda functions are mainly based on interaction with DynamoDB and are not computationally intensive.

Performance Best Practices Using Java and AWS Lambda: Initial Application

This paper is under CC BY-SA 4.0 DEED license. available on arxiv Authors: (1) Juan Mera Men´endez; (2) Martin Bartlett. Table of Links Abstract and Introduction Initial Application Performance Tests Best Practices and Techniques Combinations Discussion Related Work Conclusion and References Abstract Despite its already widespread popularity, it continues to gain adoption. More and more developers and architects continue to adopt and apply the FaaS (Function as a Service) model in cloud solutions. The most extensively used FaaS service is AWS Lambda, provided by Amazon Web Services. Moreover, despite the new trends in programming languages, Java still maintains a significant share of usage. The main problem that arises when using these two technologies together is widely known: significant latencies and the dreaded cold start. However, it is possible to greatly mitigate this problem without dedicating too much effort. In this article, various techniques, strategies and approaches will be studied with the aim of reducing the cold start and significantly improving the performance of Lambda functions with Java. Starting from a system that involves AWS lambda, java, DynamoDB and Api Gateway. Each approach will be tested independently, analyzing its impact through load tests. Subsequently, they will be tested in combination in an effort to achieve the greatest possible performance improvement. —AWS Lambda, AWS, Amazon Web Services, Java, performance, cold start, FaaS, Function as a Service, Serverless, GraalVM, Snapstart, JAVA TOOL OPTIONS Index Terms I. INTRODUCTION FOLLOWING the trend of recent years, the serverless approach [9] continues to gain popularity due to its scalability, flexibility, agility, and other well-known benefits. According to the survey conducted by Datadog in August 2023 [13], one of the most popularly used services within this architectural model is AWS Lambda (provided by Amazon Web Services) and over a 10% of Lambda functions are written in Java. [11] Furthermore Java is the primary language for 30% of professional developers. Being the performance one of the main criteria when comparing solutions, the survey shows that cold start of java lambdas are roughly twice as long as the cold starts of other languages like Node.js and Python [3], [4](which are the most used languages with AWS lambda). Indeed, it becomes evident why performance is the first aspect brought to the table when discussing the combination of Java with AWS Lambda. And rightfully so, if it’s not taken into account when building the system, it’s common to encounter time-out issues in API Gateway due to excessively high latency of the functions. Furthermore, another noteworthy benefit of improving the performance of Lambda functions is the reduction in their cost. This is because the cost model for this service is based on computing time. By reducing this time, the cost is also reduced. Thus, the primary goal of the study is to optimize the way Java code is executed on AWS Lambda and provide a set of best practices that assist this approach in being competitive in terms of performance. To achieve that goal, a series of techniques, approaches or strategies are proposed with the aim of improving function performance and mitigating cold starts: • Profile the function to see which memory configuration is better suited in terms of performance. • Use Snapstart to reduce the cold-start latency. • Choose the Arm64 Lambda architecture against the default x86 64 architecture. • Use the AWS SDK v2 for Java. • Pre-compile the source code with GraalVM to avoid initialize all classes in runtime. • Java Lambda function customization settings: JAVA TOOL OPTIONS environment variable. • Other good practices The improvement caused by each technique will be measured independently since some of them are not compatible with each other. The advantages and drawbacks of the techniques will be discussed as well. For the implementation of each technique or approach, we start with an initial, unoptimized system, which will be explained in the following section. The performance of this system has also been measured and will be used to calculate the improvements experienced relative to it. After this, the techniques will be combined, and the result will be measured.

Performance Best Practices Using Java and AWS Lambda: Abstract and Introduction

Introduction On one of the projects I worked on, there were services that used for front-end authentication and a rotated static token for back-end authentication. Despite this, the main authentication and authorization single sign-on (SSO) system was , which was used for accessing company email, the corporate VPN, infrastructure, etc. 8 Auth0 Microsoft Azure Active Directory (AD) AWS The reason Azure AD wasn’t used exclusively for user identification has more to do with the history of the project. Implementation Original Auth0 Flow Even though a hardcoded token was periodically rotated, there was still a risk that a developer or system user could access the system after leaving the company. While Auth0 was mentioned as part of our setup, my company predominantly utilized Microsoft Azure as a Single Sign-On (SSO) platform, and we aimed to integrate it further. Let's break down what occurs on the front-end part of the process: The initial step involves landing on the authentication page. After entering the login credentials, password, and two-factor authentication code, the application redirects us back to the service, appending a parameter in the URL. “?code=” The front-end library then exchanges this code for an access token. It's important to note that to exchange the code for an access token, the application must be configured to allow this without needing a client secret key. As for the communication between the front-end and back-end - despite the OAuth authentication steps described above, the application sends a predefined token, which the back-end checks for an exact match. This step is pretty standard. New Microsoft Azure AD Flow To enhance security and add greater flexibility to our system configuration — with the ultimate goal of getting rid of Auth0 and fully integrating Azure AD into  existing infrastructure — I implemented a Lambda Authorizer approach. From the AWS doc point of view: Technically, we can validate the token at the back-end, without making the infrastructure changes and adding an extra authorizing layer, which could be even more secure - instead of obtaining an access token on the front-end, we could first acquire an , which is then exchanged for an access token in the API back-end. oAuth access CODE Although this process might seem simpler, it's important to remember that we needed to accommodate an authentication flow for different services. 8 Once it’s done with a Lambda authorizer for one service, it could be applied to other services by simply setting Api Gateway + Lambda authorizer in the middle of the request. Additionally, we required functionality to validate user permissions, determining whether users are authorized to access a service or to make requests to specific endpoints of a given service. ENV/Service Reporting service Advertisement service dev ✅ ✅ prod ✘ ✅ All of this can be managed within a single Lambda function. Infrastructure Configuration and Challenges Normally, is used to point a domain's straight to the server where the website or service lives. Route 53 A record In my case, the A record was pointing to an AWS , set up to handle secure (HTTPS) connections on port 443, using an SSL certificate. The problem was that this certificate only allowed connections from websites with names like *.my-service-domain.com. Application Load Balancer (ALB) But, when I used as a middleman between Route 53 and the ALB, the ALB rejected connections because they were coming from API Gateway, not from the allowed domain name in the certificate. API Gateway To fix this, I moved an SSL certificate validation at the API Gateway level instead. However, I still needed to make sure the ALB would get connections . A big issue is that API Gateway doesn't have a fixed IP address that I could use to tell the ALB to only accept connections from this IP, by configuring a security group. only from API Gateway So, I replaced the ALB with a which can work with the API Gateway . Network Load Balancer (NLB) , VPC link Also, I had different setups for development and production environments, needing separate paths like /dev and /prod for API Gateway. But, when setting up /dev and /prod in API Gateway, it gave me URLs like this: and . https://apiId123456.execute-api.us-west-2.amazonaws.com/dev https://apiId123456.execute-api.us-west-2.amazonaws.com/prod Since Route 53 A records can't handle URLs with paths in them, I couldn't use these URLs directly. The solution was to use API Gateway's custom domains feature. This way, I could make unique URLs for each environment, like: custdomainForDev.execute-api.us-west-2.amazonaws.com custdomainForProd.execute-api.us-west-2.amazonaws.com Then, I could use these URLs as A record values in Route 53, solving my problem. Lambda Authorizer API Gateway expects a policy returned by the Lambda function that allows or denies endpoints access. Let’s have a look at the example (archived, but can be used as a template). https://github.com/amazon-archives/serverless-app-examples/blob/master/javascript/api-gateway-authorizer-nodejs/index.js 'use strict';

console.log('Loading function');

class AuthPolicy {
  constructor(principal, awsAccountId, apiOptions) {
    this.awsAccountId = awsAccountId;
    this.principalId = principal;
    this.version = '2012-10-17';
    this.pathRegex = new RegExp('^[/.a-zA-Z0-9-*]+$');
    this.allowMethods = [];
    this.denyMethods = [];

    this.restApiId = apiOptions && apiOptions.restApiId ? apiOptions.restApiId : '*';
    this.region = apiOptions && apiOptions.region ? apiOptions.region : '*';
    this.stage = apiOptions && apiOptions.stage ? apiOptions.stage : '*';
  }

  static HttpVerb = {
    GET: 'GET',
    POST: 'POST',
    PUT: 'PUT',
    PATCH: 'PATCH',
    HEAD: 'HEAD',
    DELETE: 'DELETE',
    OPTIONS: 'OPTIONS',
    ALL: '*',
  };

  setStage(stage) {
    this.stage = stage;
  }

  addMethod(effect, verb, resource, conditions) {
    if (verb !== '*' && !Object.prototype.hasOwnProperty.call(AuthPolicy.HttpVerb, verb)) {
      throw new Error(`Invalid HTTP verb ${verb}. Allowed verbs in AuthPolicy.HttpVerb`);
    }

    if (!this.pathRegex.test(resource)) {
      throw new Error(`Invalid resource path: ${resource}. Path should match ${this.pathRegex}`);
    }

    let cleanedResource = resource.startsWith('/') ? resource.substring(1) : resource;
    const resourceArn = `arn:aws:execute-api:${this.region}:${this.awsAccountId}:${this.restApiId}/${this.stage}/${verb}/${cleanedResource}`;

    if (effect.toLowerCase() === 'allow') {
      this.allowMethods.push({ resourceArn, conditions });
    } else if (effect.toLowerCase() === 'deny') {
      this.denyMethods.push({ resourceArn, conditions });
    }
  }

  getEmptyStatement(effect) {
    return {
      Action: 'execute-api:Invoke',
      Effect: effect[0].toUpperCase() + effect.substring(1).toLowerCase(),
      Resource: []
    };
  }

  getStatementsForEffect(effect, methods) {
    const statements = [];
    if (methods.length > 0) {
      const statement = this.getEmptyStatement(effect);

      methods.forEach(method => {
        if (!method.conditions || method.conditions.length === 0) {
          statement.Resource.push(method.resourceArn);
        } else {
          const conditionalStatement = this.getEmptyStatement(effect);
          conditionalStatement.Resource.push(method.resourceArn);
          conditionalStatement.Condition = method.conditions;
          statements.push(conditionalStatement);
        }
      });

      if (statement.Resource.length > 0) {
        statements.push(statement);
      }
    }

    return statements;
  }

  allowAllMethods() {
    this.addMethod('allow', '*', '*', null);
  }

  denyAllMethods() {
    this.addMethod('deny', '*', '*', null);
  }

  allowMethod(verb, resource) {
    this.addMethod('allow', verb, resource, null);
  }

  denyMethod(verb, resource) {
    this.addMethod('deny', verb, resource, null);
  }

  allowMethodWithConditions(verb, resource, conditions) {
    this.addMethod('allow', verb, resource, conditions);
  }

  denyMethodWithConditions(verb, resource, conditions) {
    this.addMethod('deny', verb, resource, conditions);
  }

  build() {
    if ((!this.allowMethods || this.allowMethods.length === 0) &&
      (!this.denyMethods || this.denyMethods.length === 0)) {
      throw new Error('No statements defined for the policy');
    }

    const policy = {
      principalId: this.principalId,
      policyDocument: {
        Version: this.version,
        Statement: [
          ...this.getStatementsForEffect('Allow', this.allowMethods),
          ...this.getStatementsForEffect('Deny', this.denyMethods)
        ]
      }
    };

    return policy;
  }
}

export const handler = (event, context, callback) => {
  const token = event.authorizationToken;

  let authContext;
  if (process.env.IS_OFFLINE) { // for local development
    authContext = JSON.parse(event.body);
  } else {
    authContext = event;
  }

  console.log('authContext', authContext);

  // [The token validation logic should be here]

  const principalId = 'user|a1b2c3d4';

  const apiOptions = {};
  const tmp = authContext.methodArn.split(':');
  const apiGatewayArnTmp = tmp[5].split('/');
  const awsAccountId = tmp[4];
  apiOptions.region = tmp[3];
  apiOptions.restApiId = apiGatewayArnTmp[0];
  apiOptions.stage = '*';

  const policy = new AuthPolicy(principalId, awsAccountId, apiOptions);

  policy.allowAllMethods();

  const authResponse = policy.build();

  authResponse.context = {
    key: 'value',
    number: 1,
    bool: true
  };

  callback(null, authResponse);
}; The result of a function test run {
  "type": "TOKEN",
  "authorizationToken": "12345678901234567890123456789012345678901234567890",
  "methodArn": "arn:aws:execute-api:us-east-1:1234567899:apiId1234/dev/POST/{proxy+}"
} will be {
  "principalId": "user|a1b2c3d4",
  "policyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "execute-api:Invoke",
        "Effect": "Allow",
        "Resource": [
          "arn:aws:execute-api:us-east-1:1234567899:apiId1234/*/*/*"
        ]
      }
    ]
  },
  "context": {
    "key": "value",
    "number": 1,
    "bool": true
  }
} that is telling the API Gateway that the user is authorized to access ANY environment using ANY request method and ANY endpoint of API. apiId1234 Optionally, the policy can return a context object containing additional information, which can be passed into the integration back-end, as per the . documentation Certainly, the above description serves as just a framework, since it lacks the token and permissions validation logic. Let’s add some now. Authorization Please note that the code below is a simplified version of an app authorization logic Adding new functionality to the Authorizer function. async function validateToken(token) {
  const res = await fetch('https://graph.microsoft.com/v1.0/me', {
    headers: {
      Authorization: `Bearer ${token}`
    }
  })

  return res.status === 200;
} is what API Gateway passes to the Lambda function. Checking if it’s valid and the session exists. token Modifying the handler’s code to handle token validation. if (await validateToken(token)) {
  policy.allowAllMethods();
} else {
  policy.denyAllMethods();
} More Fine-Grained Authorization Now, extracting groups that the user belongs to: List appRoleAssignments granted to a user GET https://graph.microsoft.com/v1.0/me/appRoleAssignments Authorization: Bearer token Response: {
  "value": [
    {
      "id": "12345667-12345678891233455678-12345667",
      "deletedDateTime": null,
      "appRoleId": "00000000-0000-0000-0000-000000000000",
      "createdDateTime": "2021-02-02T04:22:45.9480566Z",
      "principalDisplayName": "Reporting-service.dev.approved",
      "principalId": "12344567-1234-1234-123-12345678900",
      "principalType": "Group",
      "resourceDisplayName": "dxprovisioning-graphapi-client",
      "resourceId": "12344567-1234-1234-123-12345678900"
    },
    {
      "id": "12345667-12345678891233455678-12345667",
      "deletedDateTime": null,
      "appRoleId": "00000000-0000-0000-0000-000000000000",
      "createdDateTime": "2021-02-02T04:22:45.9480566Z",
      "principalDisplayName": "Advertisement-service.dev.approved",
      "principalId": "12344567-1234-1234-123-12345678900",
      "principalType": "Group",
      "resourceDisplayName": "dxprovisioning-graphapi-client",
      "resourceId": "12344567-1234-1234-123-12345678900"
    },
    {
      "id": "12345667-12345678891233455678-12345667",
      "deletedDateTime": null,
      "appRoleId": "00000000-0000-0000-0000-000000000000",
      "createdDateTime": "2021-02-02T04:22:45.9480566Z",
      "principalDisplayName": "Advertisement-service.prod.approved",
      "principalId": "12344567-1234-1234-123-12345678900",
      "principalType": "Group",
      "resourceDisplayName": "dxprovisioning-graphapi-client",
      "resourceId": "12344567-1234-1234-123-12345678900"
    }
  ]
} Let us add some code to handle that: async function getUserGroups(token) {
  const res = await fetch('https://graph.microsoft.com/v1.0/me/appRoleAssignments', {
    headers: {
      Authorization: `Bearer ${token}`
    }
  })
  const groups = res.data.value.map(group => group.principalDisplayName);
  return groups;
}; // const userGroups = await getUserGroups();
// hardcoded for visibility
const userGroups = [
  "Advertisement-service.dev.approved",
  "Advertisement-service.prod.approved",
  "Reporting-service.dev.approved"
];

userGroups.forEach(group => {
  if (group.includes("Advertisement-service")) {
    const env = group.split('.')[1]; // Extracts 'dev' or 'prod'
    policy.setStage(env);
    policy.allowMethod(AuthPolicy.HttpVerb.ALL, "/advertisement/*");
  }
});

// Explicitly allow access to dev endpoints for Reporting-service
if (userGroups.includes("Reporting-service.dev.approved")) {
  policy.setStage('dev');
  policy.allowMethod(AuthPolicy.HttpVerb.ALL, "reporting/*");
}

// Explicitly deny access to production endpoints for Reporting-service if not in the prod approved group
if (!userGroups.includes("Reporting-service.prod.approved")) {
  policy.setStage('prod');
  policy.denyMethod(AuthPolicy.HttpVerb.ALL, "reporting/*");
} An output policy has fine-grained permission. If we want to reduce authorizer hits, we can then set a policy caching from 0 - 3600 seconds (60 mins), where 0 means no cache. But it should be thoroughly thought through, taking access token expiration time into account. {
  "principalId": "user|a1b2c3d4",
  "policyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "execute-api:Invoke",
        "Effect": "Allow",
        "Resource": [
          "arn:aws:execute-api:us-east-1:1234567899:apiId1234/dev/*/advertisement/*",
          "arn:aws:execute-api:us-east-1:1234567899:apiId1234/prod/*/advertisement/*",
          "arn:aws:execute-api:us-east-1:1234567899:apiId1234/dev/*/reporting/*"
        ]
      },
      {
        "Action": "execute-api:Invoke",
        "Effect": "Deny",
        "Resource": [
          "arn:aws:execute-api:us-east-1:1234567899:apiId1234/prod/*/reporting/*"
        ]
      }
    ]
  },
  "context": {
    "key": "value",
    "number": 1,
    "bool": true
  }
} Conclusion We have discussed a flexible approach to user authorization for accessing resources, utilizing any logic and provider not natively supported by AWS. Although our implementation uses Azure AD, this method could easily accommodate any other authorization service, including a custom one. This approach offers several benefits: It provides a unified authentication and authorization point for both the front-end and back-end. There is flexibility in configuring permissions. Scalability. Once implemented for one service, it can be applied to many others.

How to Use AWS Lambda Authorizer for Flexible and Scalable Web Service Authorization

Welcome to the Startups of the Year Runner ups Interview Series. Congratulations for being one of the top 3000 startups worldwide! The HackerNoon team is thrilled to know more about you! https://hackernoon.com/startups/europe/europe-lombardy-italy What follows is an interview with , CEO and Founder at , where he talks about his experience with React Bricks and how it feels to be one of Hackernoon’s Startups of the Year 2023. Matteo Frana React Bricks Tell us how your startup is changing the world. We address the foremost challenge in content management: the divide between developer-oriented tools, such as headless CMSs, which can be overly complex for content editors, and editor-preferred tools, like visual page builders, which don't meet the technical requirements of developers or the needs of corporate websites. React Bricks bridges this gap by combining the best of both worlds. It offers the best-in-class experience to content editors while also providing a seamless and powerful development experience for React developers. inline visual editing What sets you apart from the competition? Our typically fall into two categories, when attempting to incorporate visual editing into a headless CMS: headless CMSs trying to add a visual feature as an afterthought, and no-code tools trying to eliminate the need for developers. competitors With React Bricks, to ensure the pixel-perfect brand identity is maintained. Simultaneously, for content editors, we provide a unique inline visual editing interface that is user-friendly by design, not just an additional feature. everything starts in (React) code What do you love about your team, and why are you the ones to solve this problem? We've gained a deep understanding of the industry with over 27 years of experience developing website solutions across a broad spectrum of technologies since the early days. Our collaboration with client developers and content creators, coupled with our insights into the growing popularity of React, has led us to identify a gap in the CMS market. This gives us a unique perspective on current customer needs and the skills required to develop the product that precisely meets them. If you weren’t building your startup, what would you be doing? I would likely be building another startup. I would probably be a theoretical physicist if I hadn't chosen the IT sector. At the moment, how do you measure success? What are your core metrics? We focus on customer satisfaction by closely partnering with customers to ensure their website objectives are met. By doing this successfully, revenue happens. We avoid intricate, complex, or vanity metrics, focusing on what truly matters. What goals are you looking forward to accomplishing in 2024? In 2024, we are set to launch a new version of React Bricks that supports . This will help our customers achieve optimal website performance, particularly when using Next.js. React Server Components We will introduce a suite of advanced enterprise features to assist corporations in seamlessly scaling their content management solutions. Which trend(s) are you most excited about in 2024? Share your reasons. AI-assisted creation where AI is a tool in the art-making process, but the decisions about what to create, retain, or discard are made by a human. This approach could democratize some forms of art, especially costly ones like filmmaking. It provides an opportunity for individuals who might not have had the means for self-expression. This potential for inclusivity is what excites me most about generative AI. 2023 had been another crazy year, especially in tech, with layoffs and the Generative AI takeover! Which trend are you most concerned about? Be as brief or as detailed as you like. In the long run, we might lose the brilliant creative sparks from genius creators. Imagine a world where custom movies or music are created instantaneously by AI upon request, such as or Of course, with customized video ads based on our interests, unless we opt for a premium service. “OK, Netflix, create a movie similar to Philadelphia but with a female lead” “Generate a new song in the style of Pink Floyd's album The Wall.” My concern is that we may become too comfortable with art that, while highly personalized, is dull, lacking depth and originality. The availability of content tailored to our immediate preferences and viewpoints could diminish our exposure to the unexpected and thought-provoking art that challenges us and broadens our horizons, even changing our minds. Share your biggest success so far and/or your biggest failure so far. One of our most challenging experiences occurs when CTOs are impressed with our solution, yet their company's board decides against it, favoring less effective alternatives, merely because we are an Italian company or not "large enough" even though we have been in business for more than 25 years with the same company and ownership. On the flip side, our proudest accomplishment lies in our of serving industry leaders such as The Weather Channel, , and La Banque Postale. proven track record Deel.com We would love your feedback on HackerNoon as a tech publication! How has your experience been with us? My experience with HackerNoon has been very positive. The articles are informative and cover a wide range of tech topics, which I find useful and engaging. Any words of wisdom you’d like to share with us? Starting a startup in your forties can be a wise decision. With years of accumulated experience and some savings carefully built up over the years, you're in a much stronger position to take calculated risks more judiciously than in your twenties. In your twenties, while you can take risks with more freedom due to fewer family or work responsibilities, you often have to rely on external funding from venture capitalists, and you lack experience. And in your thirties, you are typically building a family, buying a house… usually not the best time to risk with a startup idea. In your forties, you have both the experience and financial resources to invest. Independence from venture capital gives you full control over decision-making and encourages independent thinking. And since it's your own hard-earned money, you are likely to ensure that the idea you are betting on addresses a real problem in your target market. This combination of personal investment, ability to take risks, and problem-solving desire can enhance the odds of creating a successful and sustainable business. Startups of The Year is HackerNoon’s Flagship community-driven event celebrating startups that survived and thrived in 2023. 30,000 startups across 4200+ cities and six continents participated this year to be crowned the best startup in their city. See the global winners announcement . here

Meet React Bricks, Runner-up of the Startups of the Year in Lombardy, Italy

If you just want to see the code - see . github What are we going to build and why? LLMs are a new hot piece of technology that everyone is experimenting with. Managed services like OpenAI are the cheapest and most convenient way to use them. You can use their LLMs only for a small fee and for approval to train new versions of your data (in some cases). In certain applications, it is required to run an on our own. You may want to process sensitive data (medical records or legal documents) or get great quality output in a language different than English. Sometimes, you have a specialized task that doesn’t require expensive big models from OpenAI. LLM Open-source LLMs are matching the quality of big players like OpenAI but still require a lot of expensive computing resources. There are smaller models available that can run on weaker hardware and perform well enough. As an experiment, let’s deploy a smaller open-source LLM on AWS lambda. Our goal is to learn more about LLMs and docker-based lambdas. We will also evaluate performance and cost to determine if any real-world applications are feasible. For this project, we are going to use the model, a 2.7 billion parameter LLM that matches the quality of outputs from 13B parameters or more open-source models. It was trained on a large dataset and is a viable model for many applications. From my experience, it hallucinates a lot but otherwise provides useful outputs. Its size is perfect for the AWS Lambda environment. Microsoft Phi-2 We will download the model from Huggingface and run it via package (bindings to the popular llama.cpp, heavily optimized model CPU runtime). We will use a smaller, quantized version, but even the full one should fit in Lambda memory. llama-cpp-python We will create an HTTP REST endpoint via the lambda URL mechanism for a given call: PROMPT="Create five questions for a job interview for a senior python software engineer position."

curl $LAMBDA_URL -d "{ \"prompt\": \"$PROMPT\" }" \
    | jq -r '.choices[0].text, .usage' (jq command used to decode returned JSON) Provides LLM output alongside execution details: Instruct: Create five questions for a job interview for a senior python software engineer position.
Output: Questions: 

1. What experience do you have in developing web applications?
2. What is your familiarity with different Python programming languages?
3. How would you approach debugging a complex Python program?
4. Can you explain how object-oriented programming principles can be applied to software development? 
5. In a recent project, you were responsible for managing the codebase of a team of developers. Can you discuss your experience with this process?

{
  "prompt_tokens": 21,
  "completion_tokens": 95,
  "total_tokens": 116
} For this tutorial, you need to know the basics of programming, docker, AWS, and Python. I am using an M-series Mac, so as a default, we’re going to aim at deployment to ARM AWS Lambda, as it simply costs less for more performance. Instructions on how to deploy the x86 version are included. Environment setup (AWS, Docker and Python) I won’t dive into the basics here. If you don’t have the tools installed, please follow these resources. You need an AWS account and AWS CLI installed and configured. Good tutorial from Google search: . tutorial Official docs, including if you’re using SSO: AWS official tutorial. You also need docker: docker docs. As an IDE, I am using Visual Studio Code. Set up lambda function with docker locally. Let’s start with getting the local environment running. If you need more information than I included here, check out the official AWS documentation. We will store every file in a single project directory without subfolders. First, we will need a basic Python lambda function handler. In your project folder, create a file called . lambda_function.py import sys

def handler(event, context):
    return "Hello from AWS Lambda using Python" + sys.version + "!" We will also create a file, in which we specify our dependencies. Let’s start with the AWS library for interacting with their services, just as an example. requirements.txt boto3 Then, we need to specify our docker image composition in file. Comments document what each line does. Dockerfile FROM public.ecr.aws/lambda/python:3.12

# Copy requirements.txt
COPY requirements.txt ${LAMBDA_TASK_ROOT}

# Install the specified packages
RUN pip install -r requirements.txt

# Copy function code
COPY lambda_function.py ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler
CMD [ "lambda_function.handler" ] Finally, we will create a file to make our life easier when running and building the container. docker-compose.yml version: '3'

services:
  llm-lambda:
		image: llm-lambda
    build: .
    ports:
      - 9000:8080 OK, done! Make sure your docker engine is running, and type in the terminal: docker-compose up The container should be built and started. Our lambda is available under a long URL below (quirks of the official Amazon image). To test if the lambda is running, open another terminal window/tab and type in: curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}' Now you can come back to the tab you ran docker-compose in and type ctrl-c to stop the container. You can put this test code in a bash script, as seen in the project repo. Also, it will be useful to create a .gitignore file if you’re going to version control this project. Run an LLM inside a container. Now that we have a working lambda let’s add some AI magic. To run an LLM, we need to add to our . llama-cpp-python requirements.txt boto3
llama-cpp-python To build it, we need to introduce a . This is because the default Amazon docker image doesn’t include the build tools required for llama-cpp. We are doing a pip install with flags in order to use multi-threaded optimizations. docker build stage CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" The code below also includes downloading the model. Community hero, the Bloke, is sharing compressed (quantized, less computationally intensive but lower quality) versions of models, ready to download . To make use of them, we can install the huggingface CLI and run the appropriate commands. You can switch the repository ( ) and the model ( ) to whatever you like if you want to deploy a different model. here TheBloke/phi-2-GGUF phi-2.Q4_K_M.gguf RUN pip install huggingface-hub && \
    mkdir model && \
    huggingface-cli download TheBloke/phi-2-GGUF phi-2.Q4_K_M.gguf --local-dir ./model --local-dir-use-symlinks False # Stage 1: Build environment using a Python base image
FROM python:3.12 as builder

# Install build tools
RUN apt-get update && apt-get install -y gcc g++ cmake zip

# Copy requirements.txt and install packages with appropriate CMAKE_ARGS
COPY requirements.txt .
RUN CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install --upgrade pip && pip install -r requirements.txt

# Stage 2: Final image using AWS Lambda Python image
FROM public.ecr.aws/lambda/python:3.12

# Install huggingface-cli and download the model
RUN pip install huggingface-hub && \
    mkdir model && \
    huggingface-cli download TheBloke/phi-2-GGUF phi-2.Q4_K_M.gguf --local-dir ./model --local-dir-use-symlinks False

# Copy installed packages from builder stage
COPY --from=builder /usr/local/lib/python3.12/site-packages/ /var/lang/lib/python3.12/site-packages/

# Copy lambda function code
COPY lambda_function.py ${LAMBDA_TASK_ROOT}

CMD [ "lambda_function.handler" ] Let’s modify our lambda code to run LLM inference. Some more work is required to get the prompt out of the request body in the production environment. import base64
import json
from llama_cpp import Llama

# Load the LLM, outside the handler so it persists between runs
llm = Llama(
    model_path="./model/phi-2.Q4_K_M.gguf", # change if different model
    n_ctx=2048, # context length
    n_threads=6,  # maximum in AWS Lambda
)

def handler(event, context):
    print("Event is:", event)
    print("Context is:", context)

		# Locally the body is not encoded, via lambda URL it is
    try:
        if event.get('isBase64Encoded', False):
            body = base64.b64decode(event['body']).decode('utf-8')
        else:
            body = event['body']

        body_json = json.loads(body)
        prompt = body_json["prompt"]
    except (KeyError, json.JSONDecodeError) as e:
        return {"statusCode": 400, "body": f"Error processing request: {str(e)}"}

    output = llm(
        f"Instruct: {prompt}\nOutput:",
        max_tokens=512, 
        echo=True,
    )

    return {
        "statusCode": 200,
        "body": json.dumps(output)
    } OK, time to test! My docker setup has 8GB of RAM assigned, but it should run well under 4GB. Let’s rebuild the container and start it again. It will take more time as the model needs to download (~1.8GB). docker-compose up --build This time, let’s test with a real prompt. The first run will take more time as the model needs to load. curl "http://localhost:9000/2015-03-31/functions/function/invocations" \
    -d '{ "body": "{ \"prompt\": \"Generate a good name for a bakery.\" }" }' Example output: Instruct: Generate a good name for a bakery.
Output: Sugar Rush Bakery.

{
  "prompt_tokens": 15,
  "completion_tokens": 6,
  "total_tokens": 21
} Deploy to AWS Lambda With everything working locally, let’s finally deploy our LLM to AWS! To execute the deployment successfully, the following steps are required: : Determine necessary information such as AWS region, ECR repository name, Docker platform, and IAM policy file, and disable AWS CLI pager. Initial Setup : Optionally confirm that the AWS CLI is correctly configured. Verify AWS Configuration : ECR (Elastic Container Registry) Repository Management Determine if the specified ECR repository already exists. If absent, create a new ECR repository. : IAM Role Handling Check for the existence of a specific IAM role for Lambda. If not found, establish the IAM role and apply the policy. AWSLambdaBasicExecutionRole : Securely log Docker into the ECR registry using the retrieved login credentials. Docker-ECR Authentication : Utilize Docker Compose to build the Docker image, specifying the desired platform. Docker Image Construction : Label the Docker image appropriately for ECR upload. ECR Image Tagging : Transfer the tagged Docker image to the ECR. ECR Image Upload : Fetch the ARN linked to the specified IAM role. Acquire IAM Role ARN : Assess whether the Lambda function exists. Lambda Function Verification : Set parameters like timeout, memory allocation, and image URI for Lambda. Lambda Function Configuration : Lambda Function Deployment/Update For a new Lambda function, create it with defined settings and establish a public Function URL. For an existing Lambda function, update its code using the new Docker image URI. : Obtain and display the Function URL of the Lambda function. Function URL Retrieval It is a simple but lengthy process to do manually, so I’ve created a script using AWS CLI. Feel free to modify the variables and run it yourself. The script is quite complex; you can do all the steps manually in the GUI if you want, apart from pushing the container. If you are not doing this on an M-series Mac, be sure to change Linux/arm64 to Linux/amd64. #!/bin/bash

# Variables
AWS_REGION="eu-central-1"
ECR_REPO_NAME="llm-lambda"
IMAGE_TAG="latest"
LAMBDA_FUNCTION_NAME="llm-lambda"
LAMBDA_ROLE_NAME="llm-lambda-role" # Role name to create, not ARN
DOCKER_PLATFORM="linux/arm64" # Change as needed, e.g., linux/amd64
IAM_POLICY_FILE="trust-policy.json"
PAGER= # Disable pager for AWS CLI

# Authenticate with AWS
aws configure list # Optional, just to verify AWS CLI is configured

# Check if the ECR repository exists
REPO_EXISTS=$(aws ecr describe-repositories --repository-names $ECR_REPO_NAME --region $AWS_REGION 2>&1)

if [ $? -ne 0 ]; then
    echo "Repository does not exist. Creating repository: $ECR_REPO_NAME"
    # Create ECR repository
    aws ecr create-repository --repository-name $ECR_REPO_NAME --region $AWS_REGION
else
    echo "Repository $ECR_REPO_NAME already exists. Skipping creation."
fi

# Check if the Lambda IAM role exists
ROLE_EXISTS=$(aws iam get-role --role-name $LAMBDA_ROLE_NAME 2>&1)

if [ $? -ne 0 ]; then
    echo "IAM role does not exist. Creating role: $LAMBDA_ROLE_NAME"
    # Create IAM role for Lambda
    aws iam create-role --role-name $LAMBDA_ROLE_NAME --assume-role-policy-document file://$IAM_POLICY_FILE
    aws iam attach-role-policy --role-name $LAMBDA_ROLE_NAME --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
else
    echo "IAM role $LAMBDA_ROLE_NAME already exists. Skipping creation."
fi

# Get login command from ECR and execute it to authenticate Docker to the registry
aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $(aws sts get-caller-identity --query Account --output text).dkr.ecr.$AWS_REGION.amazonaws.com

# Build the Docker image using Docker Compose with specific platform
DOCKER_BUILDKIT=1 docker-compose build --build-arg BUILDPLATFORM=$DOCKER_PLATFORM

# Tag the Docker image for ECR
docker tag llm-lambda:latest $(aws sts get-caller-identity --query Account --output text).dkr.ecr.$AWS_REGION.amazonaws.com/$ECR_REPO_NAME:$IMAGE_TAG

# Push the Docker image to ECR
docker push $(aws sts get-caller-identity --query Account --output text).dkr.ecr.$AWS_REGION.amazonaws.com/$ECR_REPO_NAME:$IMAGE_TAG

# Get the IAM role ARN
LAMBDA_ROLE_ARN=$(aws iam get-role --role-name $LAMBDA_ROLE_NAME --query 'Role.Arn' --output text)

# Check if Lambda function exists
FUNCTION_EXISTS=$(aws lambda get-function --function-name $LAMBDA_FUNCTION_NAME --region $AWS_REGION 2>&1)

# Parameters for Lambda function
LAMBDA_TIMEOUT=300 # 5 minutes in seconds
LAMBDA_MEMORY_SIZE=10240 # Maximum memory size in MB
LAMBDA_IMAGE_URI=$(aws sts get-caller-identity --query Account --output text).dkr.ecr.$AWS_REGION.amazonaws.com/$ECR_REPO_NAME:$IMAGE_TAG

# Deploy or update the Lambda function
if echo $FUNCTION_EXISTS | grep -q "ResourceNotFoundException"; then
    echo "Creating new Lambda function: $LAMBDA_FUNCTION_NAME"
    aws lambda create-function --function-name $LAMBDA_FUNCTION_NAME \
        --region $AWS_REGION \
        --role $LAMBDA_ROLE_ARN \
        --timeout $LAMBDA_TIMEOUT \
        --memory-size $LAMBDA_MEMORY_SIZE \
        --package-type Image \
        --architectures arm64 \
        --code ImageUri=$LAMBDA_IMAGE_URI

    aws lambda create-function-url-config --function-name $LAMBDA_FUNCTION_NAME \
        --auth-type "NONE" --region $AWS_REGION

    # Add permission to allow public access to the Function URL
    aws lambda add-permission --function-name $LAMBDA_FUNCTION_NAME \
        --region $AWS_REGION \
        --statement-id "FunctionURLAllowPublicAccess" \
        --action "lambda:InvokeFunctionUrl" \
        --principal "*" \
        --function-url-auth-type "NONE"
else
    echo "Updating existing Lambda function: $LAMBDA_FUNCTION_NAME"
    aws lambda update-function-code --function-name $LAMBDA_FUNCTION_NAME \
        --region $AWS_REGION \
        --image-uri $LAMBDA_IMAGE_URI
fi

# Retrieve and print the Function URL
FUNCTION_URL=$(aws lambda get-function-url-config --region $AWS_REGION --function-name $LAMBDA_FUNCTION_NAME --query 'FunctionUrl' --output text)
echo "Lambda Function URL: $FUNCTION_URL" Now, let’s add proper permissions and run the script. chmod +x deploy.sh
./deploy.sh
# AWS_PROFILE=your_profile ./deploy.sh 
# if you want to use a different profile than default As a result of the script, you should se the lambda function URL in your terminal. Let’s use it for a test. PROMPT="Create five questions for a job interview for a senior python software engineer position."

curl $LAMBDA_URL -d "{ \"prompt\": \"$PROMPT\" }" \
    | jq -r '.choices[0].text, .usage' Like before, the first run will take longer. chmod +x test_remote.sh
LAMBDA_URL=your_url ./test_remote.sh You should receive back an output similar to the one at the top of the article. If you want to see the full project code, see . github Performance Cost and speed You may have noticed that I chose to run the model under the configuration: 6 core multi-threading 10GB of RAM 5-minute timeout How was the performance, and how much RAM was used? Lambda never reserves compute for you and sometimes loads your program “cold.” When you make a request, anything you need to load (like a model) needs to happen at this moment. We call this a cold start. For the prompt about five interview questions, the execution time after the cold start was 17 seconds, which is not bad at all. After that, it consistently took 9 seconds to generate an output, tested in 4 subsequent runs. The average output length was 83 tokens. So, when running hot. we achieved ~9.2 tokens/second As a comparison, community-reported speeds for OpenAI models are GPT-4 10t/s, GPT-4-turbo 48t/s, and GPT-3-turbo 50-100t/s. So we’re matching a decent but not industry-leading speed with little effort. You could run this prompt 4444 times in a free tier. Above that, . The same 1000 runs with a GPT-3.5 turbo would cost $0.2. This makes this endeavor not really cost-effective, so low cost shouldn’t be your aim if you’re going to implement this in production. it would cost $1.2 per 1000 runs Optimization Could we run the model cheaper? The model uses under 2600MB of memory consistently. We could lower the allocated resources, with a caveat that 6 core CPU is assigned only to Lambdas above ~8846MB. I ran the first test at 3500MB (3 CPU cores/threads) to see how it compares. I didn’t change the code to run only 3 threads, which had a downgrading effect. In this case, the lambda always timed out (ran over 5 minutes, which meant the request was canceled). After changing the number of threads to the proper 3, the execution almost timed out at a cold start, returning results after 4 minutes and 33 seconds; the subsequent 3 runs took 26-40 seconds, averaging 3.15 tokens/second. 1000 paid runs would cost So we got slower and paid more, not a good optimisation! With this in mind, it makes sense to pay for more memory because AWS assigns CPU resources proportionally to the reserved RAM amount. $1.45. Next steps As you can see, running an open-source language model is possible even in a regular environment like AWS Lambda. The cost and quality of output do not match industry-leading services but can serve as a use-case-specific solution. If anything, this was a great opportunity to learn about basic LLM ecosystem libraries and tools, as well as strengthen your AWS knowledge. Some possible next steps if you want to keep exploring and bringing this project closer to production-ready: Try to run a different (bigger) model or a less compressed version of Microsoft Phi-2. Create a CDK config to automatically deploy the model and create resources. Create a GitHub Action to validate the project and run ./deploy.sh. Try a different serverless platform to run the model. Subscribe to my newsletter and follow me on social media. I may write about some of these topics, let me know in the comment if you’re interested in any of them! If you bump into any issues let me know as well. What do you think about this way of deploying LLMs? Subscribe to my profile by filling in your email address on the left, and be up-to-date with my articles! Don't forget to follow me on Twitter and subscribe to my blog’s for more tips and insights! @ horosin_ newsletter If you don't have Twitter, you can also follow me on . LinkedIn Also published . here

Deploying Open-Source Language Models on AWS Lambda

Serverless NestJS Automation on AWS Lambda

Joey looked out his front window, contemplating Coit Tower. He thought of Artie who had changed his life forever on the crookest street in the world. It felt like a dream when he thought back. Was that day real when he met Artie on this curvy brick street? No matter how unreal those times seemed now, he knew they had happened. His very expensive house on the exclusive brick section of Lombard Street was proof for sure. When Artie gifted him this incredible view of the bay, Joey’s life was forever changed. In the thirty-plus years he had been there, a day did not pass he did not think of and thank Artie for the view. He had met Artie in North Beach. Artie was from out of town. Joey found his British accent charming. Artie had a room at the Fairmont atop Nob Hill while Joey was renting a flat in the basement of an ancient building. Joey’s mind wandered to joyful times spent gallivanting up and down Nob Hill. Artie was a successful writer who traveled the world. Joey and Artie fell deeply in love during those halcyon days. Artie was just in San Francisco for a summer conference. Joey was surprised to hear him say he lived in Sri Lanka. However, any time he was on the West Coast he would come to San Francisco to visit Joey. The last time had been back in 1993. “Joey come visit me on Lombard Street,” said Artie on the telephone “Lombard Street. You are not staying on Nob Hill anymore?” Joey asked. “No, I am over on Lombard Street, 1040 Lombard Street. Come on over Joey I have a surprise for you.” Artie’s voice was pregnant with mystery. Joey was on the most crooked street in the world within the hour. The front window looked out on the bay. He could see the Oakland Bay Bridge and Treasure Island in the distance. San Francisco was an especially magical place back in those days. A place where a poor kid from Turlock might be gifted a dream home on a magical street in a magical city. “What!?” Joey was incredulous that such a gift could be given or would be given. He loved Artie and Artie loved him. Yes, they had a special bond, but a house seemed like too much. “I don’t have any money for a down payment Artie? I cannot afford the mortgage payment even if I did.” “It doesn’t matter Joey. I am giving it to you. It is already paid for. I just have to put the deed in your name,” Artie explained calmly. “I don’t understand. Why?” Joey asked Joey’s revelry was broken when he noticed some rather large SUVs parked in the street. He did not recognize them. Ever since the street had been closed to vehicles other than locals, strange autos on the brick road were conspicuous. Eight men in dark suits exited the vehicles as Joey watched. “NSA!” The men shouted at Joey They surrounded Joey. He was hustled through his front door. Within seconds they were in his living room. “We are here to deliver this,” said one of the suits. He thrust an envelope into Joey’s hand. No one had shown Joey any ID, but he had seen weapons. Joey was unsure what to do. He sat for a moment and then opened the envelope. In it, he found a note in a strange language. A language he never thought he would actually need, Harrapan. Artie had taught Joey Harrapan all those years ago. He had never heard anyone speak about the language. It was a mystery to Joey why Artie had been so adamant that Joey learn the language as part of getting the deed to the house. “This IS the mortgage Joey and mortgages aren’t easy to pay,” Artie explained. Now Joey was looking at a note partially written in Harrapan. The English language portion of the note was about the Clarke-Asimov Treaty. Joey was not unfamiliar with the Clarke-Asimov Treaty. Joey was a big science fiction buff. He had read Asimov’s Foundation series multiple times. He particularly loved Arthur C. Clarke’s Childhood’s End story. He recalled the stories about how Arthur C. Clarke and Isaac Asimov developed an amicable rivalry. It was legendary within science–fiction fandom. The rivalry eventually led to the Clarke–Asimov Treaty. The Clarke–Asimov Treaty stated that each author would refer to the other as the world’s greatest writer in his specialty — and refer to himself as merely second–best. Under these terms, Asimov would crown Clarke as the best science–fiction author ever, while Clarke would anoint Asimov as the greatest science writer. Of course, each could publicly crown himself a close second to the other. The two reputedly created the pact during a shared cab ride. The only time either acknowledged this treaty was in Clarke’s dedication to his book, Report on Planet Three. In that book, Clarke wrote: “In accordance with the terms of the Clarke–Asimov treaty, the second-best science writer dedicates this book to the second-best science-fiction writer.” Joey always wondered if his “friend” Artie was THE Arthur C. Clarke. He sure looked like all of the photos…kind of. Artie was a big fan of those early writers of science fiction. Artie used to always talk about how coincidental it was that Isaac Asimov and Robert Heinlein had worked at the Philadelphia Naval Yard during World War II. He hinted at some top-secret work Asimov had been exposed to during World War II. Sometimes if he drank a little too much he would say Asimov told Arthur C. Clarke something “special” about the Philadelphia Experiment. In the morning though, he would deny or claim to remember nothing. Joey often asked him about UFOs. Artie would laugh. “Aliens”, he would say with a chuckle, “We don’t need no stinking aliens!” “Well?” asked one of the suits who was not pointing a gun at him startling him from his reflections on the Clarke-Asimov Treaty. Joey looked up. There were three other large men holding large assault weapons aimed in his direction. He felt scared. He struggled to remember anything Arthur had taught him in Harrapan. “I cann…can’t remember,” he pleaded. The suit in front of Joey raised his hand. Joey cowered fearing he was about to be struck. The suit signaled for his compatriots to lower their weapons. “Don’t be afraid. We’re here from your government,” he explained softly. “Don’t worry. My name is Jack. We are with the NSA. You are not in trouble. We just need your help,” he continued. Joey looked like a deer in headlights. He stared at the symbols on the piece of paper. His mind went back to his days with Artie. He remembered Artie talking about the language of a long-lost ancient civilization. The Harrapan script was a mix of letters, but also of glyphs which meant a whole word or concept. He remembered Artie trying to teach him, but the fact that some symbols were whole words while others were simply letters was difficult to tease out of the text. “Look, my name is John Stick. We are with the NSA. We need you to read this text in the native tongue. Intelligence reports you may be the only person on the planet who can read this piece of paper and speak the phrase in the native tongue,” explained the suit as he flashed an official-looking identification card. “What? Why me? I cannot read this,” Joey said weakly. “ I cannot speak Harrapan.” “Harrapan! Yes, that is exactly what you need to do,” shouted the suit named John excitedly. Joey’s mind raced. The symbols did look familiar somehow. He thought about Artie. He smiled. The two had many beautiful moments on the beaches around San Francisco. Joey heard himself whisper, “Jo-E korgin qa Ahrrt-E. Artie loves Joey.” “That’s it!” shouted a woman with a small device in her hand. Suddenly, windows started breaking in the front of the house. It was not just the windows. Everything was breaking. The room was being peppered with bullets. “Vykhod!” The man who called himself John Stick ordered and directed. Everyone moved to the back of the house except Joey. All of the suits and the woman fled out the back as soon as there was a moment of silence. Joey just laid behind his couch shaking as he watched them run to his garage. “FBI! FBI! Hands up. Hands up. Show yourself now or be shot!” There were many loud voices and orders being shouted. “He’s here. He’s here! Joey stood up from behind the couch cautiously. He touched his chest and other parts of his body incredulous there was no blood or injury. He started to cry. “You are not hurt! What did you tell them?” “Tell who? They were the NSA,” Joey defended the fact he had translated the note. He didn’t know who was who anymore. “Look. I am Paul Jerome from the FBI,” said a tall red-haired man with a beard. “Those people were not from the NSA. They were Russians. They blew up the Georgia Guide stones last year.” “Really?” Joey was now interested because he had wondered why these had been blown up. He had planned on visiting them last year while making his annual trip to visit his father in South Carolina. However, traffic in Atlanta had been bad so he skipped the trip. A month later, they were partially blown up and then mysteriously demolished with heavy equipment by local authorities within hours. “Yes, the Russians were looking for the time capsule underneath the guide stones.” “Did they find it?” Joey asked inquisitively “Yes, and that note they gave you was what was in it! We need you to tell us what you read there. Your voice and your voice alone can decode the video squirreled away somewhere on the Ethereum blockchain. The location of the fabled Black Pyramid of the Urals is revealed in the video. This is the real reason for the War in Ukraine. Allegedly, the Ukrainians somehow discovered the location of this ancient structure and discovered its secrets. The Black Pyramid contains a cache of ancient technology. This technology is capable of generating enormous power. These are essentially weapons of mass destruction…in the eyes of many.” Alarm bells went off in his head. He knew he must cooperate or end up in Guantanamo. The next words confirmed this for him. “This is a matter of NATIONAL SECURITY Joseph,” the FBI agent said matter-of-factly. “You are the ‘oracle’. Your voiceprint of this message will complete the smart contract and release the video.” “Yessir,” Joey responded obediently. “It was Harrapan sir. This is what I said.” Joey paused and then said “Jo-E korgin qa Ahrrt-E. Artie loves Joey.” He spoke directly into the device thrust into his face. Also published here.

Oracle on Lombard Street: The Harrapan Revelation

The keystroke dynamics that are used in this article’s machine learning models for user recognition are behavioral biometrics. Keystroke dynamics uses the distinctive way that each person types to confirm their identity. This is accomplished by analyzing the 2 keystroke events on Key-Press and Key-Release — that make up a keystroke on computer keyboards to extract typing patterns. The article will examine how these ML Models can be used in real-life situations to predict an user. The previous article described how we can train 3 ML models, on a set of +880 keyboard inputs from 100 users that were asked to write the same text 12 times. https://hackernoon.com/the-application-of-ml-models-in-user-recognition-via-keystroke-dynamics?embedable=true The apps that we’ll be using to simulate a real life situation. The Architecture : In order to test real life scenarios we will use + . Sandbox Postman Flask : When we want to move this application into production we can simply replace the bits of the Flask code with code for an . Production AWS Lambda Flask is a web-framework we can use to replicate the Lambda + API Gateway. The Lambda is an serverless web-service hosted in AWS, that can run event driven code written in various languages, however for this app we will use python. The data flow is: We load the header with identification data and the model we want to use, this is basically our In the we will also add the (Prod only) json payload. header authorization key We do HTTP POST method request to Flask App/API Gateway API Gateway checks the authorizaiton key (api key) and if it accepts the request, sends the to the Lambda for processing payload The FlaskApp/Lambda runs the underlying code What we get in the return is the ID of the predicted user if event is successful, else we get an error message body As previously mentioned all the code you can find on the Github project page: https://github.com/BogdanAlinTudorache/KeystrokeDynamics The Postman request Raw {
 "info": {
  "_postman_id": "c62ddbda-e487-432f-998a-4dfc9313f0fa",
  "name": "BogdanTudorache",
  "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
 },
 "item": [
  {
   "name": "predict_user",
   "request": {
    "method": "POST",
    "header": [],
    "body": {
     "mode": "raw",
     "raw": "{\n    \"Model\": \"RF\",\n   \"HT\": {\n        \"Mean\": 48.43,\n        \"STD\": 23.34\n    },\n    \"PPT\": {\n        \"Mean\": 120.43,\n        \"STD\": 37.41\n    },\n    \"RRT\": {\n        \"Mean\": 124.43,\n        \"STD\": 45.34\n    },\n    \"RPT\": {\n        \"Mean\": 132.56,\n        \"STD\": 47.12\n    }\n}",
     "options": {
      "raw": {
       "language": "json"
      }
     }
    },
    "url": {
     "raw": "http://127.0.0.1:5000",
     "protocol": "http",
     "host": [
      "127",
      "0",
      "0",
      "1"
     ],
     "port": "5000"
    }
   },
   "response": []
  }
 ]
} You can download the file from GitHub. Body {
    "Model": "RF",
   "HT": {
        "Mean": 48.43,
        "STD": 23.34
    },
    "PPT": {
        "Mean": 120.43,
        "STD": 37.41
    },
    "RRT": {
        "Mean": 124.43,
        "STD": 45.34
    },
    "RPT": {
        "Mean": 132.56,
        "STD": 47.12
    }
} The Flask App / Lambda App If we want to sandbox test the process before moving everything to AWS we must replicate an end-to-end call using Flask. How to start the Flask App? In the GitHub, you will find the full version of flask_lambda_function.py and in any IDE you just need to run the script as it will automatically start the flask app. In my case, I am using IntelliJ Idea so I simply run the script (click right → run): Testing a real-life scenario Once the script is started in the bottom part of the IDE you will see the python console informing that the webservice has started on localhost and port 5000 (i think this is default however this can also be configured). Now every time we do a HTTP request POST , we will have trigger the flask app, simulating a real life scenario. POST request: Python console reply: Postman response: Code DeepDive @app.route('/', methods=['GET', 'POST'])
def index():
    # Bellow code should be uncommented when running in AWS:Lambda
    # above should be commented as well as flask import + app definition
# def lambda_handler(event, context):
    """
     Lambda handler: When a request hits the API gateway linked to this lambda_function this is the function that gets
     called.
     The request data is passed as the event variable which is a dictionary object, in this case it the json of
     the POST request from which we extract the body details
    """

    # Parses the details from the POST request: extracts model and input data
    # Based on model it imports the trained model from local
    # Outputs the predicted user based on input data
    try:
        prediction = functions.predict_user(request.get_json())
        # Below code should be uncommented when running from AWS, above should be commented.
        # prediction = functions.predict_user(event)
        return jsonify({'statuscode': 200,
                        'status': 'success',
                        'predicted user': str(prediction)
                        })
    except Exception as e:
        return jsonify({'statuscode': 400,
                        'status': 'error',
                        'message': str(e)}) The above code is set to flask-app however by commenting and uncommenting certain lines, you can easily switch to Lambda. The structure is very simple, we’re simply calling another function, which is predict_user() and providing as input the json body from the POST request. predict_user() def predict_user(event):
    """ Gets the input details from the body of the POST request and returns the predicted user
    """
    # Print the event for debugging purposes
    print(event)

    # Check if the message has the correct body structure
    if ['Model', 'HT', 'PPT', 'RRT', 'RPT'] == list(event.keys()):
        print(f"Model is:", event['Model'])

        if event["Model"] == "SVM":
            # Load the trained SVM model from the joblib file
            model_path = os.path.join(basedir, 'models', 'svm_model.joblib')
            model = joblib.load(model_path)

        elif event["Model"] == "RF":
            # Load the trained Random Forest model from the joblib file
            model_path = os.path.join(basedir, 'models', 'rf_model.joblib')
            model = joblib.load(model_path)

        elif event["Model"] == "XGBoost":
            # Load the trained XGBoost model from the joblib file
            model_path = os.path.join(basedir, 'models', 'xgb_model.joblib')
            model = joblib.load('model_path')

        # Extract the features from the event dictionary
        features = [
            event['HT']['Mean'],
            event['HT']['STD'],
            event['PPT']['Mean'],
            event['PPT']['STD'],
            event['RRT']['Mean'],
            event['RRT']['STD'],
            event['RPT']['Mean'],
            event['RPT']['STD']
        ]

        # Make a prediction using the loaded model and the extracted features
        prediction = model.predict([features])

        # Return the predicted user
        return prediction[0] This function is simplistically written, depending on the model parameter we decide to load the specific pre-trained ML model. With the pre-trained model we do the user prediction and return it to the requester. ( ) see last Postman response photo The training of the models has been done in of this article, . part I see above link If you like the article and would like to support me, make sure to: 🔔 Follow me Bogdan Tudorache 🔔 Connect w/ me: | LinkedIn Reddit

Keystroke Dynamics — Predicting the User — Lambda App

কীস্ট্রোক ডায়নামিক্স — ব্যবহারকারীর পূর্বাভাস দেওয়া — ল্যাম্বডা অ্যাপ

Tastendynamik – Vorhersage des Benutzers – Lambda-App

Dinámica de pulsaciones de teclas: predicción del usuario: aplicación Lambda

Dynamique de frappe — Prédire l'utilisateur — Application Lambda

कीस्ट्रोक डायनेमिक्स - उपयोगकर्ता की भविष्यवाणी - लैम्ब्डा ऐप

キーストロークのダイナミクス — ユーザーの予測 — Lambda アプリ

키 입력 역학 — 사용자 예측 — Lambda 앱

Dinâmica de pressionamento de tecla - Prevendo o usuário - Aplicativo Lambda

Динамика нажатия клавиш — прогнозирование действий пользователя — приложение Lambda

Tuş Vuruşu Dinamiği — Kullanıcıyı Tahmin Etme — Lambda Uygulaması

Động lực gõ phím — Dự đoán người dùng — Ứng dụng Lambda

击键动力学 — 预测用户 — Lambda 应用程序

The Go-Getter: A Story That Tells You How to be One by Peter B. Kyne is part of the HackerNoon Books Series. You can jump to any chapter in this book . I here I Mr. Alden P. Ricks, known in Pacific Coast wholesale lumber and shipping circles as Cappy Ricks, had more troubles than a hen with ducklings. He remarked as much to Mr. Skinner, president and general manager of the Ricks Logging & Lumbering Company, the corporate entity which represented Cappy's vast lumber interests; and he fairly barked the information at Captain Matt Peasley, his son-in-law and also president and manager of the Blue Star Navigation Company, another corporate entity which represented the Ricks interest in the American mercantile marine. Mr. Skinner received this information in silence. He was not related to Cappy Ricks. But Matt Peasley sat down, crossed his legs and matched glares with his mercurial father-in-law. " have troubles!" he jeered, with emphasis on the pronoun. "Have you got a misery in your back, or is Herbert Hoover the wrong man for Secretary of Commerce?" You "Stow your sarcasm, young feller," Cappy shrilled. "You know dad-blamed well it isn't a question of health or politics. It's the fact that in my old age I find myself totally surrounded by the choicest aggregation of mental duds since Ajax defied the lightning." "Meaning whom?" "You and Skinner." "Why, what have we done?" "You argued me into taking on the management of twenty-five of those infernal Shipping Board freighters, and no sooner do we have them allocated to us than a near panic hits the country, freight rates go to glory, marine engineers go on strike and every infernal young whelp we send out to take charge of one of our offices in the Orient promptly gets the swelled head and thinks he's divinely ordained to drink up all the synthetic Scotch whiskey manufactured in Japan for the benefit of thirsty Americans. In my old age you two have forced us into the position of having to fire folks by cable. Why? Because we're breaking into a game that can't be played on the home grounds. A lot of our business is so far away we can't control it." Matt Peasley leveled an accusing finger at Cappy Ricks. "We never argued you into taking over the management of those Shipping Board boats. We argued me into it. I'm the goat. You have nothing to do with it. You retired ten years ago. All the troubles in the marine end of this shop belong on my capable shoulders, old settler." "Theoretically--yes. Actually--no. I hope you do not expect me to abandon mental as well as physical effort. Great Wampus Cats! Am I to be denied a sentimental interest in matters where I have a controlling financial interest? I admit you two boys are running my affairs and ordinarily you run them rather well, but--but--ahem! Harumph-h-h! What's the matter with you, Matt? And you, also, Skinner? If Matt makes a mistake, it's your job to remind him of it before the results manifest themselves, is it not? And vice versa. Have you two boobs lost your ability to judge men or did you ever have such ability?" "You're referring to Henderson, of the Shanghai office, I dare say," Mr. Skinner cut in. "I am, Skinner. And I'm here to remind you that if we'd stuck to our own game, which is coast-wise shipping, and had left the trans-Pacific field with its general cargoes to others, we wouldn't have any Shanghai office at this moment and we would not be pestered by the Hendersons of this world." "He's the best lumber salesman we've ever had," Mr. Skinner defended. "I had every hope that he would send us orders for many a cargo for Asiatic delivery." "And he had gone through every job in this office, from office boy to sales manager in the lumber department and from freight clerk to passenger agent in the navigation company," Matt Peasley supplemented. "I admit all of that. But did you consult me when you decided to send him out to China on his own?" "Of course not. I'm boss of the Blue Star Navigation Company, am I not? The man was in charge of the Shanghai office before you ever opened your mouth to discharge your cargo of free advice." "I told you then that Henderson wouldn't make good, didn't I?" "You did." "And now I have an opportunity to tell you the little tale you didn't give me an opportunity to tell you before you sent him out. Henderson a good man--a crackerjack man--when he had a better man over him. But--I've been twenty years reducing a tendency on the part of that fellow's head to bust his hat-band. And now he's gone south with a hundred and thirty thousand taels of our Shanghai bank account." was "Permit me to remind you, Mr. Ricks," Mr. Skinner cut in coldly, "that he was bonded to the extent of a quarter of a million dollars." "Not a peep out of you, Skinner. Not a peep. Permit me to remind that I'm the little genius who placed that insurance unknown to you and Matt. And I recall now that I was reminded by you, Matthew, my son, that I had retired ten years ago and please, would I quit interfering in the internal administration of your office." you "Well, I must admit your far-sightedness in that instance will keep the Shanghai office out of the red ink this year," Matt Peasley replied. "However, we face this situation, Cappy. Henderson has drunk and gambled and signed chits in excess of his salary. He hasn't attended to business and he's capped his inefficiency by absconding with our bank account. We couldn't foresee that. When we send a man out to the Orient to be our manager there, we have to trust him all the way or not at all. So there is no use weeping over spilled milk, Cappy. Our job is to select a successor to Henderson and send him out to Shanghai on the next boat." "Oh, very well, Matt," Cappy replied magnanimously, "I'll not rub it into you. I suppose I'm far from generous, bawling you out like this. Perhaps, when you're my age and have a lot of mental and moral cripples nip you and draw blood as often as they've drawn it on me you'll be a better judge than I of men worthy of the weight of responsibility. Skinner, have you got a candidate for this job?" "I regret to say, sir, I have not. All of the men in my department are quite young--too young for the responsibility." "What do you mean--young?" Cappy blazed. "Well, the only man I would consider for the job is Andrews and he is too young--about thirty, I should say." "About thirty, eh? Strikes me you were about twenty-eight when I threw ten thousand a year at you in actual cash, and a couple of million dollars' worth of responsibility." "Yes sir, but then Andrews has never been tested----" "Skinner," Cappy interrupted in his most awful voice, "it's a constant source of amazement to me why I refrain from firing you. You say Andrews has never been tested. Why hasn't he been tested? Why are we maintaining untested material in this shop, anyhow? Eh? Answer me that. Tut, tut, tut! Not a peep out of you, sir. If you had done your Christian duty, you would have taken a year's vacation when lumber was selling itself in 1919 and 1920, and you would have left Andrews sitting in at your desk to see the sort of stuff he's made of." "It's a mighty lucky thing I didn't go away for a year," Skinner protested respectfully, "because the market broke--like that--and if you don't think we have to hustle to sell sufficient lumber to keep our own ships busy freighting it--" "Skinner, how dare you contradict me? How old was Matt Peasley when I turned over the Blue Star Navigation Company to him, lock, stock and barrel? Why, he wasn't twenty-six years old. Skinner, you're a dodo! The killjoys like you who have straddled the neck of industry and throttled it with absurd theories that a man's back must be bent like an ox-bow and his locks snowy white before he can be entrusted with responsibility and a living wage, have caused all of our wars and strikes. This is a young man's world, Skinner, and don't you ever forget it. The go-getters of this world are under thirty years of age. Matt," he concluded, turning to his son-in-law, "what do you think of Andrews for that Shanghai job?" "I think he'll do." "Why do you think he'll do?" "Because he ought to do. He's been with us long enough to have acquired sufficient experience to enable him--" "Has he acquired the courage to tackle the job, Matt?" Cappy interrupted. "That's more important than this doggoned experience you and Skinner prate so much about." "I know nothing of his courage. I assume that he has force and initiative. I know he has a pleasing personality." "Well, before we send him out we ought to know whether or no he has force and initiative." "Then," quoth Matt Peasley, rising, "I wash my hands of the job of selecting Henderson's successor. You've butted in, so I suggest you name the lucky man." "Yes, indeed," Skinner agreed. "I'm sure it's quite beyond my poor abilities to uncover Andrews' force and initiative on such notice. He does possess sufficient force and initiative for his present job, but--" "But will he possess force and initiative when he has to make a quick decision six thousand miles from expert advice, and stand or fall by that decision? That's what we want to know, Skinner." "I suggest, sir," Mr. Skinner replied with chill politeness, "that you conduct the examination." "I accept the nomination, Skinner. By the Holy Pink-toed Prophet! The next man we send out to that Shanghai office is going to be a go-getter. We've had three managers go rotten on us and that's three too many." And without further ado, Cappy swung his aged legs up on to his desk and slid down in his swivel chair until he rested on his spine. His head sank on his breast and he closed his eyes. "He's framing the examination for Andrews," Matt Peasley whispered, as he and Skinner made their exits. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Peter B. Kyne (2004). The Go-Getter: A Story That Tells You How to be One. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/12257/pg12257-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

Mr. Alden P. Ricks, known in Pacific Coast wholesale lumber and shipping circles as Cappy Ricks

The first story I wrote about Pepe back in May was very popular, and I wasn’t sure if I would be writing about PEPE again… but here we are. A lot has happened in the past few days, and it looks like this story has now reached a major plot twist. Since I’m picking the story up from where I left it last time, I strongly suggest you here so you have more context and it makes better sense. read the first part So, let’s see how things have been since then. A few days after my Pepe article was published, there was another that came out. That one painted everything in much darker colors and brought to attention things that, if true, were disturbing. Medium post Things like: Prior to hitting it big with $PEPE, the individual behind the curtain is guilty of countless consecutive hard rugs. This token launch was about as unfair as humanly imaginable. There was no equal opportunity present, there was no decentralization in the building; there was only thirty or more wallets controlled and manipulated by the same entity. shortly after the launch the dev blacklists the earliest of buyers that weren’t him or his friends As far as I know, this was also the first time the public found out about who one of the Pepe team members was: i.e., Zachary Testa. As you can see, the accusations were quite serious, especially if you look back and compare it to how $Pepe was marketed and positioned (fair launch, distribution, etc.…), which, in turn, was probably one of the biggest reasons people aped in. So were we all fooled (again)? And if that’s the case - why didn’t that article gain any significant traction? Why did it get forgotten and/or ignored? I don’t really know, but there may be several reasons: It was published by an anon account. The piece was extremely emotional. The author was shilling another coin at the end. No matter the reasons though, the article was not taken seriously, and pretty much everyone moved on. The community was strong, the X Spaces kept going, the memes kept flowing, the coin got several centralized exchange listings, and at one point, $PEPE even hit a $1.6B market cap. And just as I had already noticed, in the very beginning, the main community action was happening on X and X Spaces… Those official Telegram and Discord channels full of “wen moon, wen lambo” guys kind of did not matter. The team seemed not to be involved either. They never came to Spaces, and as far as I know, there was no engagement with the community apart from periodic broadcasting from their accounts. Even the that took place all around the world in June was completely self-organized by the community. PepeFest So you could say, that at this point, the community was self-sufficient, and the vibes were very positive. Many who got in early made quite generous profits… but obviously, the ones who made the most were the anon dev team. And What Happens When ICO/IDO Founders Succeed? What’s the First Thing They Do? Yup. They Go and Buy a Brand-new Lamborghini. It’s a well-known classic move. I’ve seen it happen before. I’m Surprised that Lambo is still a definition/demonstration of success among crypto bro-s… but oh well… And that’s exactly what Zachary did too - he bought a purple Lamborghini priced over $800k. Most of the community found out about it from Pauly. Seems like he knew the team from when he introduced them to Binance to help with the coin’s listing on the exchange. While Zachary drove his shiny new Lambo, Pauly (in his distinguished style) kept insisting that the artist (Matt Furrie) whose work they profited off should be paid. Unfortunately, that never happened, and up to now, to my knowledge, the team has not made any donation to Matt Furie (there is a separate story about this, but I will tell it in a different piece). Up to now, the community did not worry much about anything. The common thought process probably was: What is there to worry about? The smart contract is renounced, so the devs can’t change anything anymore. Everything is community-driven so it feels like the team is kind of irrelevant at this point. And the last: Just as I was predicting in the previous article the community was indeed tracking the multisig wallet. And here we are - we have reached the big plot twist! The multisig wallet was touched. To be more precise, here is exactly what happened: On August 24, 2023, the multi-sig wallet reduced the threshold to just 2/8 signatures. This means that any transaction approvals will need only 2 signatures out of 8 to execute. That by itself felt strange and alarming, and obviously, the community started tweeting about it and discussing the topic immediately. Please note that there is a big misconception that those signatures are always from different people (obviously that should be the case, otherwise what’s the point, right?). But the setup can’t check if it’s different people or not. It can check the list of wallets and see that 2 out of the list of eight have signed… But theoretically, both wallets can belong to the same person, right…? This means that from the community perspective, the number really doesn’t matter much, and you either trust the team or you don’t. So right after that change, the team started transferring $PEPE to centralized exchanges. We all know what happens after that by now, right? Yup - a huge selloff and price drop. Oh, wait… but didn’t they say… Oh dear… what a shock. Yet another anon dev team lied to its community. As if it never happened before… right? How Many Lambos Have Degens Already Funded By Now?... All this happened with no transparency and zero communication from the Pepe team. And what happens when there is an information vacuum? Usually FUD… The community gathered in Spaces and kept discussing… many followed wallet movements and charts… and X was full of findings, speculations, emotions, and what not… That’s when Pauly published a . thread about Zachary In his thread, he too (like in the Medium post from May) mentioned Pepe insider distribution wallets. The team finally came out with on the 3rd day: on August 26, 2023. the announcement The whole blame was put on “3 ex-team members” who were “bad actors led by big egos and greed” who allegedly were the ones who moved roughly $15m USD worth of Pepe to centralized exchanges. But there is still 10 trillion $PEPE remaining in the multi-sig and here is the promise: The remaining 10 trillion tokens left in the multi-sig will be transferred out of the old multi-sig and into a new wallet where they will safely rest until a use or burn arises. I have been in negotiation with the owners of some web domains and usernames that I am looking into acquiring for $PEPE, and when I am complete with any of these prospective purchases or donations of $PEPE from the multi-sig, I will burn the remainder of these multi-sig tokens. Once all is set and complete, I would like $PEPE to be left in a position where it can stand and thrive as strong as ever in a truly decentralized and anti-fragile state. Together, we are all $PEPE. Do you trust this statement? Seems like not many do as this poll suggests that the community wants those burnt immediately! The story is still unraveling. The community wants justice and keeps meme-ing Zachary and his purple Lambo. Pauly has tweeted out tons of information about Zachary, and by now, pretty much all his family. There may already be a report filed with the SEC (but I have no proof). Some exited their positions and/or reallocated them into other coin/s. I even saw this emerge in the middle of this all. Note: I have taken out the links, so you don’t go there. I’ve only seen this tweet and am not endorsing anything related to it. I think $PEPE’s future depends entirely on the community. They will either be able to do what Phunks did back in the days of similar highly turbulent times, when they fought for survival … or not. Only time will tell. Disclaimer: I had bought a small amount of $PEPE for the culture back when I was writing the 1st article and still HODL most of it now. Also published here

Stories, News & Mentions about LYMB

Lymb’s Stories on HackerNoon