We are on the brink of the next industrial revolution and Oden is using our technologies to usher in the new era of manufacturing

Oden Technologies

Introducing the GPT OpenData Explorer The past few days after the from OpenAI that users will be able to create custom versions of ChatGPT or GPTs, I decided to do an experiment creating one myself. Something that has always been bugging me is the difficulty of finding information of high quality when searching most topics on the web regarding Greece or, frankly, any search using the Greek language, especially compared to English search results. The aforementioned, combined with the generally always tricky task of getting information from trustworthy sources, led to the creation of the** **. This GPT is mainly powered by the Open Data published by the Greek central government, local authorities, and public bodies at . announcement OpenData Explorer GPT https://data.gov.gr OpenData Explorer GPT The big advantage that an chatbot like this GPT can provide is natural language understanding and how that can . LLM simplify the process of people searching for information Journalists, students, or anyone else can access (and soon properly visualize) data regarding education, the environment, health, transport, crime, etc. By making open data more accessible, we can increase their impact on society. A few examples: • Increase transparency and accountability: Users can check the trend of how many people take public transport and compare it to the available fleet; they can monitor the quality of broadband internet • • OpenData Explorer currently supports only data that derive from , but if positive results come out of this attempt, the goal would be to integrate it with open data from other countries as well. This would enable intriguing combinational prompts and provide access to a much greater audience. It's still worth mentioning, like any other piece of technology, but even more so with such a new one, it is still very crucial to check the validity of any AI-generated response. If you've read this far, the fun visual part comes now. Below, I will share some of the example prompts you can give the OpenData Explorer GPT and compare it to plain ChatGPT. I would love people to try it and test out its potential and, of course, receive feedback about it. Raise awareness on environmental issues: Users can check the number of fires, the electricity consumption, and the renewable energy sources used all over the country Social impact: Users can observe unemployment trends, crime rates, drunk driving accidents https://data.gov.gr Let me know what you think! As a disclaimer for this GPT: for a lot of dates, there is no data GPT is limited to the available datasets from data.gov.gr it's not good yet at doing basic operations with the results (e.g., adding them) if you don't guide it properly graph creation is not the best, either some of the responses are too big for it to handle Open Data If you are interested in Open Data, below are a few observations and interesting points that I discovered during my investigation of Open Data in Greece. According to https://data.europa.eu Open (Government) Data refers to the information collected, produced or paid for by the public bodies (also referred to as Public Sector Information) and made freely available for re-use for any purpose. It is worth mentioning that the quality of this data is very important, and a key part of that is providing a machine-readable friendly format (e.g., via API technologies). For : https://data.gov.gr • the API access option to the data is very useful • documentation and instructions on how to use the API are lacking • there are only 52 datasets available • a lot of the datasets are not updated (e.g., data for forest wildfires stops on 2018-12-31 or electricity consumption data is provided only for a few days of 2020) • on the site, you can find the number of people who have used the API, and I was surprised to see that OpenData Explorer was, in a lot of cases, the first to make requests to it Greek Open Data: • there is more available open data from Greek organizations, but it's not in , and in a lot of cases, it is in non-friendly formats (such as PDFs) • Greece ranked 28th (out of 35) in Open Data Maturity in 2022, according to • according to the "Greece 2.0" plan, a total budget of 5.227.840,00 euros will be invested, having started from 2023 and until 2025, to the project of "Open data and enhancing participation" see If you want to learn more about the state of open data in Greece, I highly recommend reading* *, an article by . data.gov.gr https://data.europa.eu/en/publications/open-data-maturity/2022#country-overview https://greece20.gov.gr/wp-content/uploads/654.SUB5_ANOIKTA_DEDOMENA_EDYTE_5190853.pdf The slow death of open data in Greece Eliza Triantafyllou I'm also following the initiatives of , which contributes to the development of Open Technologies in Greece. ΕΕΛ/ΛΑΚ GFOSS

OpenData Explorer GPT: Unlocking Information through AI

The development and deployment of new ‘AI’ based tools continues to gather momentum. As this rapidly becomes big-business, I believe we need to be doing much more to ensure that, unlike the big business ideas of the past, we build in essential safeguards to the technology and our ways of using that technology before the opportunity arises for serious harm to humans or humanity. I should make it clear that I’m not talking about doomsday scenarios here like killer robots wiping out the world. Whilst these scenarios do bear consideration, I do not believe they are the most immediate threat. I’m talking about your everyday mundane use of AI to automate decisions and slowly take people out of the conversation and decision loop. This is where we need to have strong rules, governance, and ethics [1]  that ensure our AI tools are always safe when used, can be deployed without causing harm, and that a tool is able to explain its reasonable and show its course of action is appropriate. Are you ready to be a responsible AI practitioner? Here are six simple principles you can follow, that will help act and deploy your AI tools responsibly. Each principle is based on a cornerstone of ethical behaviour and should feel familiar – the impact is in bringing them altogether and staying true to them. Six Behaviours for Responsible Practitioners 1. Act Honestly An easy one to start with. Make sure everyone impacted knows that they are interacting with an AI-driven system or that such a system has made an automated decision on behalf of the business. Don’t pretend your chatbot is a human. Don’t imply a person has carefully weighed a decision or has understood the details of an individual case. 2. Be Trustworthy Use privacy enhancing technology, like homomorphic encryption, differential privacy, and federated learning. These minimise the leaking of private information whilst still providing the statistics and patterns of interest. Additionally, properly deployed, they can help us combine data across different privacy jurisdictions. Technology is only one part of the solution. You must also have strict policies in place limiting how information is used and when new permission needs to be sought. However, privacy is only one variable of the trust equation. It is equally important that your models can be relied upon when learning from fully aggregated data or completely non-personal datasets. Always be clear about when and how you are sourcing information and clear about how you verify every piece of information in the model. 3. Offer Respect There is a question of whether driving a particular interaction, decision, or recommendation through an AI without other options is most appropriate to a particular individual. Should we have to accept an automated result that isn’t validated or checked by a person. When deploying AI into our processes we should consider how can respect the sovereignty and beliefs of others and make it so that the application of AI is optional, replaceable, or at least under the supervision of a person. 4. Operate Transparently It’s important to be clear how information is processed and how any system reaches a decision or recommendation. It’s important that, as far as possible any system can explain how it has reached a decision and the way in which it has combined factors to deliver a result. Being able to show sources and their reliability is also important. Avoiding black boxes where reasoning cannot be followed is essential in any responsible decision making or recommendation process. This is a challenge using current commercial large-scale models and so investing in building your model using open technologies and securing professional external support to establish and train the models may be required. 5. Show Compassion How do you cope with exceptions? Will your system need to interact with people with special or difficult circumstances? Is it possible to recognise when a person needs support that’s outside of system capabilities and direct them to more appropriate help? 6. Maintain Integrity Let people know what your ethical principles are and your intentions in this. Decide how you will follow the other principles discussed and publish that intent. Let people see that you are abiding by them and that they can have trust in you and your actions. This also means being prepared to be open when things go wrong and be prepared to put them right. Ensure you act in a way that is fair to those using or impacted by the system. Understand it’s limitations and biases. Let people know what they are instead of glossing over them. Have a plan to mitigate bias in your model. When it’s not practical to easily overcome limitations or mitigate bias, don’t allow the model to be deployed where there is risk of harm. References [1] These ideas are paraphrased and adapted from Isaac Asimov’s commentary on how his own Laws of Robotics had analogues for every tool, whether robotic or not

Creating a Culture of Responsible AI

Navigating the Future: Enhancing Bitcoin's Scalability and Sustainability Introduction to Bitcoin's Scalability and Environmental Challenges Bitcoin, the first and most well-known cryptocurrency, has revolutionized the financial world since its inception in 2009. Offering a decentralized, secure, and transparent transaction mechanism, Bitcoin has paved the way for the development of numerous other cryptocurrencies and blockchain projects. However, its journey has not been without challenges. Among the most significant hurdles are scalability and environmental sustainability. The core technology of Bitcoin, the Proof-of-Work (PoW) blockchain, demands extensive computational power, leading to high energy consumption and slow transaction processing times. These limitations have sparked intense debate within the crypto community and hindered Bitcoin's broader adoption, particularly in decentralized applications and social media platforms. The Layer-2 Revolution: A Scalable and Sustainable Solution To address these challenges, the blockchain community has turned to innovative solutions known as Layer-2 protocols. These protocols operate on top of the existing blockchain (Layer-1), aiming to enhance scalability without compromising security or decentralization. Social Network: Pioneering Bitcoin's Layer-2 Staking Protocol One of the most notable advancements in this space is the of the Social Network, a decentralized Bitcoin Layer-2 staking protocol. By launching its testnet, Social Network has introduced a to improving Bitcoin's scalability and environmental footprint. The protocol leverages recent technological upgrades and partnerships to provide a scalable, cost-efficient, and less energy-intensive solution. development groundbreaking approach Collaborations and Innovations A key to Social Network's success is its with industry leaders such as ChainSafe, Halborn Security, Threefold, BloxRoute, and LaunchNodes. These partnerships aim to create a secure and efficient BTC staking platform that supports decentralized social networks, leveraging the security and trustlessness of the Bitcoin blockchain. collaboration Beyond Social Network: The Broader Landscape of Layer-2 Solutions Layer-2 solutions are not limited to staking protocols. Other technologies, such as Rootstock and Stacks, extend Bitcoin's functionality through smart contracts and decentralized applications (dApps), further enhancing the network's scalability and interoperability with Ethereum. These introduce new functionalities to the Bitcoin ecosystem, improving transaction speeds and offering new opportunities for blockchain applications. developments The Path Forward: Envisioning a Greener, More Connected Digital Future The unveiling of Layer-2 protocols, such as the Social Network, heralds an era of transformation for the Bitcoin network, emphasizing not just technological advancement but a commitment to ecological responsibility. These innovations signal a pivotal shift towards creating a Bitcoin ecosystem that is both scalable and environmentally conscious. The significance of these developments extends beyond the immediate community to hint at a future where blockchain technology is integral to a sustainable global digital economy. Bridging Technology and Sustainability Layer-2 solutions like Social Network are more than just technical feats; they are a testament to the blockchain community's resolve to foster a digital realm that prioritizes sustainability alongside scalability. This evolution is critical for the wider acceptance and integration of Bitcoin into decentralized applications, paving the way for a future where blockchain's potential is fully unleashed without sacrificing our planet's health. Catalyzing Transformation Across the Digital Landscape The ripple effect of Layer-2 advancements stretches far beyond the confines of the Bitcoin network. By addressing the pressing issues of scalability and environmental impact, these technologies open the door to an array of possibilities for decentralized platforms and applications. The emergence of these solutions could revolutionize how we interact with digital and financial systems, making them more accessible, efficient, and, importantly, sustainable. A Collaborative Journey Towards Innovation The path to realizing the full potential of blockchain technology is a collaborative endeavor, requiring the collective effort of developers, researchers, and industry visionaries. It is through this synergy that the community can navigate the complex web of challenges that lie ahead, driving forward innovations that balance technological advancement with environmental stewardship. Pioneering a Sustainable Digital Financial Ecosystem The transition towards Layer-2 solutions marks a critical step in Bitcoin's evolution, offering a blueprint for overcoming its inherent challenges. By enhancing the network's scalability and minimizing its ecological footprint, the blockchain community is laying the groundwork for a digital financial ecosystem that is inclusive, efficient, and, above all, sustainable. This journey is laden with both opportunities and obstacles, yet the promise of innovation and collaboration lights the way towards a brighter, greener future. The advancements in Layer-2 technologies are not just reshaping the Bitcoin network; they are redefining the boundaries of blockchain's role in a sustainable digital economy. As we stand on the brink of this new era, the collective endeavor of the blockchain community will undoubtedly propel us towards a future where digital innovation and environmental sustainability go hand in hand, heralding an age of digital finance that is as green as it is global. Don’t forget to like and share the story! This author is an independent contributor publishing via our brand-as-author program. Be it through direct compensation, media partnerships, or networking, the author has a vested interest in the company/ies mentioned in this story. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR Vested Interest Disclosure:

Social Network Leads the Charge in Bitcoin's Layer-2 Evolution: A Greener Future Awaits

সোশ্যাল নেটওয়ার্ক বিটকয়েনের লেয়ার-২ বিবর্তনে দায়িত্ব পালন করে: একটি সবুজ ভবিষ্যত অপেক্ষা করছে

Soziales Netzwerk ist führend in der Layer-2-Entwicklung von Bitcoin: Eine grünere Zukunft erwartet Sie

La red social lidera la evolución de la capa 2 de Bitcoin: le espera un futuro más verde

Le réseau social mène la charge dans l'évolution de la couche 2 de Bitcoin : un avenir plus vert vous attend

सोशल नेटवर्क बिटकॉइन के लेयर-2 विकास में अग्रणी भूमिका निभाता है: एक हरित भविष्य की प्रतीक्षा है

ソーシャルネットワークがビットコインのレイヤー2進化の先頭に立つ: より環境に優しい未来が待っている

소셜 네트워크는 비트코인의 레이어 2 진화를 주도합니다: 보다 친환경적인 미래가 기다리고 있습니다

A rede social lidera a evolução da camada 2 do Bitcoin: um futuro mais verde o aguarda

Социальная сеть лидирует в эволюции Биткойна второго уровня: нас ждет более экологичное будущее

Sosyal Ağ, Bitcoin'in 2. Katman Evriminde Liderlik Ediyor: Daha Yeşil Bir Gelecek Bekliyor

Mạng xã hội dẫn đầu trong quá trình phát triển lớp 2 của Bitcoin: Một tương lai xanh hơn đang chờ đợi

社交网络引领比特币第 2 层进化：更绿色的未来正在等待

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Ke Hong, Tsinghua University & Infinigence-AI; (2) Guohao Dai, Shanghai Jiao Tong University & Infinigence-AI; (3) Jiaming Xu, Shanghai Jiao Tong University & Infinigence-AI; (4) Qiuli Mao, Tsinghua University & Infinigence-AI; (5) Xiuhong Li, Peking University; (6) Jun Liu, Shanghai Jiao Tong University & Infinigence-AI; (7) Kangdi Chen, Infinigence-AI; (8) Yuhan Dong, Tsinghua University; (9) Yu Wang, Tsinghua University. Table of Links Abstract & Introduction Backgrounds Asynchronized Softmax with Unified Maximum Value Flat GEMM Optimization with Double Buffering Heuristic Dataflow with Hardware Resource Adaption Evaluation Related Works Conclusion & References 7 Related Works Large language model inference acceleration has gained significant attention in recent research, with several notable approaches and techniques emerging in the field. DeepSpeed [9] is a comprehensive engine that optimizes both the training and inference phases for LLMs. It achieves robust inference performance through kernel fusion and efficient GPU memory management, with a particular focus on optimizing memory usage for KVcache. vLLM [11] improves GPU memory utilization by efficient memory management techniques and the PageAttention method, leading to increased maximum batch sizes and elevating the upper limit of inference performance. FlashAttention [18, 19] optimizes the self-attention computation process during the prefill phase through improved parallelism and workload distribution. FlashDecoding [13] is an extension of FlashAttention and enhances the parallelism through spliting K and V , supporting efficient self-attention computation for long sequence during the decode phase. FasterTransformer [33] and OpenPPL [12] implement large model inference engines using C++ to reduce overhead resulting from kernels scheduling, compared to Python implementations. They also employ memory management techniques and kernel fusion to achieve efficient LLM inference. TensorRT-LLM [14] is built upon the TensorRT [38] and the FasterTransformer [33] engine (C++) and incorporates cutting-edge open-source technologies such as FlashAttention [18, 19]. Additionally, it enhances its ease of use by providing the Python API.

FlashDecoding++: Faster Large Language Model Inference on GPUs: Related Works

The introduction of AI, and ChatGPT in particular, has created a real hassle across industries. Not only do tech specialists think of exciting opportunities the new technologies open to us, but they also fear job replacements. The World Economic Forum (WEF) showed in their that between 2023 and 2028, around 44% of workers’ skills will be disrupted. And AI is exactly the game changer that comes into play. study The role of in software development continues to increase, bringing more efficiency to the daily routine, contributing to smart solutions, and helping to automate repetitive and manual tasks. AI AI can really help to decrease the gap between programmers, between the experienced specialists and the entry level candidates. But does it really change developers’ salaries and replace their jobs? Going Through Developers’ Salary Trends: Artificial Intelligence in Software Development Last year brought many AI solutions and tools, such as GitHub Copilot, AlphaCode, and aiXcoder, whose usage quickly spread among developers. They help to decrease the time for production and improve the analysis and productivity. Some tools offer code review capabilities; others have options for integration with other platforms. The fact is undeniable – such tools optimize the coding process. Naturally, with all AI’s opportunities, the question arises: will AI take over software engineering? Will new codes be written by AI, while humans will lose their jobs? The answer is “no”. Developers will still be in high demand, and their salaries, on the contrary, have all the chances to grow. Here is why. What Changes Can the Usage of AI in Software Development Bring? With numerous interactions with clients and stakeholders in Outstaff Your Team, we see that the demand for software developers is still high, even if tech companies consider the usage of AI tools. At the moment, the most common tasks that AI tools can cover include: Automated code review Code generation Bug detection Automated testing Documents generation Vulnerability detection If previously the difference between entry-level specialists and experienced professionals was perceptible, now with , the gap tends to decrease. Besides, the usage of artificial intelligence in software development helps to save time and reduce efforts in cumbersome tasks, making developers more efficient and allowing them to focus on more meaningful tasks. AI for software development Risks and Benefits of AI for Software Development The usage of AI in software development has moved the specialists’ work to a new level, bringing with it higher speed, better performance, and innovation. Developers can now accelerate their development process and proactively address any issues. With AI’s predictive analytics, they can now better predict the development outcomes and suggest process improvements. But, with all the existing risks, currently the future of work is dependent on automation rather than specialists’ replacement. AI models can still cause biases, leading to inaccurate predictions. The usage of large datasets can involve data privacy concerns and security vulnerabilities. Moreover, the AI for software development still needs validation and cannot take decisions and work autonomously as human professionals. So, the question, “Will coding become redundant?” doesn’t have grounds and should not raise our concerns. Instead, the developers should focus on their upskilling and on their expertise in AI in software development, which will, in turn, make them even more competitive. What Influences Software Developers’ Salaries? Even though the demand for software developers doesn’t tend to decrease, the salaries go through a number of fluctuations that will impact their amount in the end. The most widespread reasons include: A candidate with a degree in computer science or engineering may have a higher salary than a candidate who only finished a couple of months of courses. Education and qualifications. We’re used to tech giants having bigger budgets than small companies. However, depending on the industry, startups can sometimes offer big salaries and better working conditions. Industry and company size. The difference in pay between developed and developing countries may seem enormous until you start counting your spending and taxes. Geographic location. If the supply fills in the gaps of demand, the salary is less likely to go too high. Market demand. Specialists with niche skills and many years of experience usually get higher compensation than entry-level professionals. Skills and experience. Depending on the location and company size, the salary of software developers varies between $120-170K in the US and $50-70K in Europe. Other countries from around the world join the demand for IT specialists, with such countries as South Africa, New Zealand, or India offering from $40K to $60K. Unless specialists work remotely, they have little to no impact on geographic location and market demand. However, they can perfectly manage their skills and experience. And this is exactly the area where a salary can be increased. As demand for AI in software development continues to grow, specialists can upskill and obtain higher posts. Skills of the Future The appearance of artificial intelligence in software development created stiff competition among tech specialists. The more they follow the trends, the higher competencies they will have. From my experience, I saw many tech companies, both small and large, that demand a combination of hard and soft skills from tech candidates. In 2024, more and more companies will focus on mobile applications and AI implementation, which means that the following skills will be needed: Mobile development frameworks, such as Angular, React, and Django. Knowledge of cloud platforms, such as Azure and . AWS Database management tools (ex. MySQL, DevOps, Salesforce). AI and machine learning skills, including PyTorch and TensorFlow frameworks. Project management skills and Agile methodologies. Will software engineering be automated, and will developers remain competitive without continually improving all these skills? Certainly not. Software engineering is one of the first areas that will continue to evolve, and being able to quickly adapt is one of the main criteria in the current market. AI That Changes Our Working Environment AI rapidly transforms both the developing processes and the compensation of specialists. The number of companies that aim to implement AI tools continues to increase while professionals turn such tools to their advantage, saving time and reducing manual tasks. Though many are concerned that AI will reduce their salaries or take off their jobs, such tools have shown exceptional results in automation, control, development, and deployment. The accessibility to AI gave specialists numerous possibilities for development and upskilling. If the tools are used smartly and add value to the routine work, they will only fulfill developers’ expertise. If developers move with the times and learn AI technologies, they will remain competitive in any field and location.

AI in Software Development: Developers’ Salary Shifts and Trends

Introduction As LLM technologies gain more mainstream adoption and the ecosystem begins to mature, organizations are beginning to recognize the limits and costs of using LLM technologies. Many enterprises, originally enthusiastic about the application of , have abandoned centralized initiatives, pursuing instead a strategy of encouraging decentralized efforts to incorporate services like ChatGPT and Claude into their workflows. LLM technologies There are several reasons for this phenomenon. Lack of LLM expertise, MLOps requirements, and reliance on specialized GPU infrastructure are all barriers to implementing large-scale AI initiatives. Of these, however, the thorniest issue is the reliance on GPUs. In this article, we will discuss the specific difficulties posed by GPU reliance, explore a potential solution, and look at an exciting example from one of the trailblazing companies working in this area. GPU Availability as Limitations to LLMs Most publicly available and highly performant models, such as GPT-4, Llama 2, and Claude, all rely on highly specialized GPU infrastructure. GPT-4, one of the largest models commercially available, famously runs on a cluster of 8 A100 GPUs. Llama 2’s 70B model, which is much smaller, still requires at least an A40 GPU to run at a reasonable speed. This level of GPU requirement practically forecloses the possibility of running these models locally - a A100 GPU, assuming you can find a seller, costs close to $25,000. Once you obtain the GPUs, you need specialized skills to set up and maintain the servers. Very few organizations would be willing to incur such an outlay to experiment with LLM technologies. To solve this problem, several startups and cloud providers have developed extensive PaaS offerings. Some services like Replicate, which I have used in past articles and projects, allow users to rent GPU servers and pay for compute time used. Other providers, like OpenAI and Anthropic, offer their models as a per-token API, further abstracting away infrastructural complexities. However, these services require data to be sent to an external network, which makes using these services a non-starter for privacy-conscious organizations. Additionally, many of these services suffer from shortages during demand spikes as GPU usage overtakes availability, making them unreliable options for production-critical workloads. Additionally, GPU time, no matter how they are charged for, is expensive for large compute tasks - the companies that own and operate these GPUs need a return on their investment, after all. While these costs are nearly negligible for experimental use cases, commercial use cases often require embedding large contexts, fine-tuning, or multi-shot examples. These costs represent a significant barrier to adoption, especially for organizations with large datasets or those that lack the financial resources of large U.S. firms. In a previous article, we explored parameter compression as one strategy to reduce dependence on GPUs. In today’s article, we will explore another exciting technique called quantization. Before we dive into the exploration, however, we might want to learn a little bit about quantization first. Quantization (Optional Reading) In this section, we will briefly go over the basics of quantization. However, if you’re simply looking for a way to run powerful LLMs locally on your computer, you can feel free to skip this section for now and come back later. LLMWare, the company whose technology we will be using today, has built some amazing tools that let you get started with quantized models without having to get into the nitty-gritty behind it all. C/C++ implementations What is Quantization? Quantization is a technique that seeks to reduce the computational and memory requirements of running an LLM by using lower-precision numerical types. Many popular open-source models, such as Llama, Falcon, and Alpaca, use PyTorch as the underlying framework. By default, PyTorch models use 32-bit floating points, meaning a single parameter takes up 32 “bits” in the GPU memory. Quantization aims to replace these parameters with 16-bit floating points, 8-bit integers, or even 4-bit integers. Successful quantization leads to dramatic improvements in computational speed and reductions in memory usage, meaning large models become runnable on lower-end GPUs, embedded graphic chips, or even CPUs. This idea has been around for a while - PyTorch itself has added support for 16-bit floating points and model compilations as the technology matured, but progress has been slow due to early design decisions in the PyTorch framework. Does Quantization Degrade Performance? At this point, it’s natural to wonder, wouldn’t this severely degrade the accuracy of the model? The short answer is yes, but only if you do it carelessly. Every optimization comes with inherent trade-offs, but with some specialized techniques, researchers have been able to squeeze incredibly stable performance out of highly quantized models. While we won’t go into extreme technical detail, let’s go over the broad strokes of the most common strategies being used right now. If you want to learn more, you can find out more about it in a from HuggingFace. guide Calibrated Quantization During the quantization process, a calibration dataset is run through the model. The value of each parameter is recorded, and the range is used to determine how the parameters get quantized. Assuming the calibration dataset is representative of the inputs that the model will encounter, this results in improved accuracy of the resulting model. Quantization-Aware Whereas Calibrated Quantization happens after training, Quantization-Aware Training tries to optimize the model during training. While the model is training, the activations are put through a “fake quantization,” simulating errors that will likely be introduced by the quantization process. The model is then able to adapt to the errors, resulting in a more robust model that can specifically adapt to the potential distortions. Llama.cpp and GGUF While PyTorch quantization and optimizations have been long blocked by framework design, two recent open-source technologies broke through these barriers and made quantization technologies much more accessible to the general public. Let’s briefly cover them below. Llama.cpp Llama.cpp was a project by Georgi Gerganov to port the Llama model into C/C++. This got rid of the complexity introduced by PyTorch, and the native implementation allowed quantization to be implemented directly. Therefore, the resulting model could run with up to 4-bit integer quantization, allowing high-parameter-count Llama models to be run without a specialized GPU. The project has since been extended by the community to include a roster of open-source models, including popular ones like Falcon and Mistral. GGUF GGUF is Llama.cpp’s file format for storing and transferring model information. Quantized models are stored in this format so that they can be loaded and run by the end-user. GGUF is the successor format to GGML and aims to improve on GGML by providing more extensibility, backward compatibility, and stability while still allowing for rapid development. The development of a universal file format opened the door for the open-source community to extend the Llama.cpp to optimize other models, and innovators like TheBloke and LLMWare have been working over the last several months to miniaturize popular open-source models. LLMWare’s Quantized Dragon Model In today’s example, we will be using open-source libraries and quantized models provided by LLMWare, which provides convenient tools for quickly building specialized RAG workflows. Who is LLMWare? LLMWare, a generative AI company specializing in the legal and financial industries, has been actively involved in the quantization community. As I’ve written before, their focus on privacy-conscious sectors makes them a natural candidate for experimenting with and innovating in miniaturization technologies. Previously, I wrote about their RAG-optimized BLING models that squeeze incredible performance out of 1 to 3 billion parameter models for specialized tasks like contract review and financial analysis. While most open-source models with such parameter counts tend to be only useful for toy problems, LLMWare is able to generate production-ready performance out of these models by training them for narrowly targeted tasks. These miniaturized models are then able to run without an external GPU, allowing for increased privacy and scalability. What is Dragon? Dragon is a collection of LLMs that can be thought of as more powerful versions of their BLING cousins. The original intent of Dragon was to train a higher parameter model using the same instruction fine-tuning techniques, providing an option to users that need more performance and have access to lower-end GPUs. The added parameter count resulted in more powerful models that could leverage larger context windows and generate more complex outputs, but required the user to have more specialized hardware, such as a GPU-embedded laptop or a cloud compute container with a GPU attached. However, they still represented an improvement over the extremely large models, which would require waiting for access to scarce A40 or A100 GPUs. Quantized Dragon, the Best of Both Worlds Given the above, it is easy to see why quantization gave a significant boost to LLMWare’s suite of AI tools. With quantization, a user could run Dragon-tier models on the same environment as BLING models, allowing for much more powerful analysis on commodity computers. Over the course of the last month, LLMWare has published quantized versions of several Dragon models. Today, we will evaluate LLMWare’s Dragon model built on top of Llama with a legal analysis RAG problem and compare it with a similar BLING model. Interested reachers can also explore other models - a Mistral-based model and a Yi-Based model are available from LLMWare at the time of this writing. Additionally, has made running inferences on Llama.cpp models a breeze with their tight integrations with the ctransformers library, which allows gguf models to be swapped seamlessly with PyTorch based models. LLMWare We will use a Macbook Air with M1 Chip for this experiment, meaning we will only be using widely available hardware for this exercise. Testing Out Quantized Dragon Remember that in my previous article, we built a RAG application focused on legislation search. We used vector search to quickly search through several large legislations, found sections relevant to our question about Qualified Opportunity Zone Partnership Interest, and ran the question through a BLING model. In today’s article, we will run the same question through LLMWare’s quantized Dragon model and determine if it performs better than BLING models. In order to focus on model comparison and to reduce the amount of prior knowledge required, we will do a lot of the PDF parsing and vector search manually. This has the added benefit of making the problem artificially harder for the model - LLMWare’s default embedding search chunks the source material to about 1000 tokens, but handling the parsing manually allows us to bump the context up to around 3000 tokens. This will help us clearly demonstrate the difference between the Dragon and BLING models. However, you should be able to easily integrate with the rest of the LLMWare’s ecosystem if you want to leverage their tools by following the setup steps from my last article on LLMWare. In fact, if you simply replace the name of the BLING models with the quantized Dragon model from this article, everything should run seamlessly. Without further ado, let’s get started! First, let’s import the required dependencies: import sklearn
import sklearn.metrics # for cosine similarity
from llmware.prompts import Prompt
import time
import os
from openai import OpenAI
from PyPDF2 import PdfReader 

client = OpenAI() # the library now loads the key automatically as an environment variable. We can now load the PDF. In the previous example, we loaded several large legislations, but for today, we will only focus on the PDF version of the Tax Cuts and Jobs Act of 2017. reader = PdfReader([path to PDF of tax cuts and jobs act]) Now we can generate the embeddings for each page: embeddings = []
for pg in reader.pages:
    text = pg.extract_text() 
    embeddings.append(client.embeddings.create(
        input=text,
        model="text-embedding-ada-002"
    ).data[0].embedding) Let’s also generate the embeddings for the question we’re going to ask: question = 'What is a qualified opportunity zone partnership interest?'
q_embed = client.embeddings.create(
        input=question,
        model="text-embedding-ada-002"
    ).data[0].embedding With the embedding in hand, we can perform a vector search. Because our search space is small, we can just do this manually. cos_sim = [(idx, sklearn.metrics.pairwise.cosine_similarity([e], [q_embed])[0][0]) for idx, e in enumerate(embeddings)] Now we can take the most relevant page (which is index 132 or page 133 if you want to verify the results): most_relevant = sorted(cos_sim, key=lambda x: x[1], reverse=True)[0][0] And with that, we have come to the most crucial step. We will instantiate an LLMWare Prompter object with the quantized Llama Dragon model. The Prompter class is key here because it handles the prompt engineering for us and makes sure that our prompt is consistent with the structure of Dragon’s training data. The prompt class also automatically handles the llamacpp binding, so you can use the quantized Dragon model exactly like other models. model_name = "llmware/dragon-llama-7b-gguf"
prompter = Prompt().load_model(model_name)

response = prompter.prompt_main(question, context='\n\n'.join([reader.pages[132].extract_text()]), 
                                prompt_name="default_with_context", temperature=0.3) Wait a little while, and you should see the function call return. Now print the results: print(response['llm_response']) And you should see something like the following: • A capital or profits interest acquired by the qualified opportunity fund after December 31, 2017, from the partnership solely in exchange for cash;
•As of the time such interest was acquired, the partnership was a qualified opportunity zone business (or, in the case of a new partnership, it was being organized for purposes of being a qualified opportunity zone business);
•During substantially all of the qualified opportunity fund's holding period for such interest, the partnership qualified as a qualified opportunity zone business. This is quite a good answer! For comparison, let’s see how a BLING model would perform on the same problem. One of the issues we can expect is that the large context size can “overwhelm” a lower-parameter model and lead to a less informative answer. In my previous experiments, the sheared llama 2.7b was one of the best performers for this problem, so I decided to use that as the representative of the BLING models. model_name_2 = "llmware/bling-sheared-llama-2.7b-0.1"
prompter2 = Prompt().load_model(model_name_2)

response = prompter2.prompt_main(question, context='\n\n'.join([reader.pages[132].extract_text()]), 
                                prompt_name="default_with_context", temperature=0.3) After some processing, you should see something like this. A qualified opportunity zone partnership interest is a capital or profits interest in a domestic partnership if such interest is acquired by the qualified opportunity fund after December 31, 2017, from the partnership solely in exchange for cash. The response is still good but misses some of the details captured by the Dragon model. Specifically, the answer misses the holding period requirement and the new business case. This conforms to our expectations about the lower-parameter models’ difficulty with processing larger contexts. Interested readers can extend this experiment by using even lower parameter models or increasing the size of the given context. You should see the effect become increasingly more pronounced, after which the model will give a short, garbled answer. From this experiment, it should be clear that quantized Dragon models are able to outperform lower-parameter models for their intended use cases without noticeably compromising the model’s accuracy. And with that, we have used a quantized model to solve a real-world use case and learned about its performance characteristics in the process! Conclusion Today, we explored the exciting field of LLM quantization and looked at how companies like LLMWare are taking advantage of these developments to enhance their specialized language models. As I’ve argued before, miniaturization represents one of the most promising paths to the widespread adoption of AI technologies. By combining specialization, fine-tuning, and quantization, innovators in the AI space can create scalable and performant models that solve real-world problems. You can find LLMWare’s RAG framework on and their DRAGON and BLING models on LLMWare’s . Github Hugging Face repository By the way, I am working on an exciting project that seeks to use language AI and miniaturization to revolutionize education in the developing world. We’re working with incredible activists and educators worldwide, and we’re working to bridge the global digital divide. If you’d like to learn more about my project or simply want to talk about exciting developments in the LLM space, please do not hesitate to reach out to me on either or . Github LinkedIn

Run Llama Without a GPU! Quantized LLM with LLMWare and Quantized Dragon

একটি GPU ছাড়া লামা চালান! এলএলএমওয়্যার এবং কোয়ান্টাইজড ড্রাগনের সাথে কোয়ান্টাইজড এলএলএম

Führen Sie Llama ohne GPU aus! Quantisiertes LLM mit LLMWare und Quantized Dragon

¡Ejecute Llama sin GPU! LLM cuantificado con LLMWare y Quantized Dragon

Exécutez Llama sans GPU ! LLM quantifié avec LLMWare et Quantized Dragon

बिना जीपीयू के लामा चलाएं! एलएलएमवेयर और क्वांटाइज्ड ड्रैगन के साथ क्वांटाइज्ड एलएलएम

GPU なしで Llama を実行しましょう! LLMWare と Quantized Dragon を使用した Quantized LLM

GPU 없이 Llama를 실행해보세요! LLMWare 및 Quantized Dragon을 사용한 양자화 LLM

Execute o Llama sem GPU! LLM Quantizado com LLMWare e Dragão Quantizado

Запустите Llama без графического процессора! Квантованный LLM с LLMWare и Quantized Dragon

Lama'yı GPU olmadan çalıştırın! LLMWare ve Quantized Dragon ile Quantized LLM

Chạy Llama mà không cần GPU! LLM lượng tử hóa với LLMWare và Quantized Dragon

无需 GPU 即可运行 Llama！使用 LLMWare 和 Quantized Dragon 进行量化 LLM

The challenge of breaking into the tech often revolves around the dilemma of required experience but struggling to gain that experience without an opportunity. To address this issue, I’ve manually curated a diverse list of 21 tech internships from open source to more specified programs by various companies. I hope this compilation will help you gain valuable experience in the tech industry and allow you to take the first step toward your dream role! 1. Outreachy Outreachy provides internships in open-source and open science. Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they are living. 👉 https://www.outreachy.org/ 2. Open Source Promotion Plan OSPP has united with domestic and international open-source communities to propose project tasks for the development and maintenance of important open-source software. 👉 https://summer-ospp.ac.cn/ 3. Winter of Code GDSC NSEC hosts its annual celebration of the collaborative spirit of communities and open-source technologies, the Winter of Code (WoC). Each year, tentatively in the months of Jan-Feb, they are home to contributors from across the world, looking to contribute to projects dedicated to the event by participating. 👉 https://winterofcode.com/ 4. OpenCode IIITA A month-long program for students to start their journey in the world of open-source. The Only requirement is an enthusiastic heart to learn. 👉 https://opencodeiiita.github.io/ 5. Open Summer of Code A 4-week summer program in July provides Belgian-based students the training, network, and support necessary to transform open innovation projects into powerful real-world services. 👉 https://osoc.be/ 6. Google Summer of Code A global, online program focused on bringing new contributors into open-source software development. GSoC Contributors work with an open-source organization on a 12+ week programming project under the guidance of mentors. 👉 https://summerofcode.withgoogle.com/ 7. Google Season of Docs Google Season of Docs provides support for open-source projects to improve their documentation and gives professional technical writers an opportunity to gain experience in open-source. Together we raise awareness of open source, of docs, and of technical writing. 👉 https://developers.google.com/season-of-docs/ 8. GitHub Octernships Program The GitHub Octernships program connects students with industry partners in paid professional experiences and mentorship on open source and software development projects. 👉 https://education.github.com/students/octernships 9. Foundation for Public Code Internship program This program offers you the opportunity to work on high-value public code projects used around the world and improve the role of technology in society. You’ll help to make public-purpose software and policy – used for example in cities and other public organizations – more open and collaborative. 👉 https://publiccode.net/careers/intern.html 10. FOSSASIA Internship Program In the program, FOSSASIA is looking for people who would like to work on the project they choose continuously. Different from GSoC in the internship it is not only about a specific project proposal. They look for participants who are interested in advancing the project and solving bugs or adding features that are required to bring the project forward. 👉 https://docs.google.com/forms/d/e/1FAIpQLScp8h5SIPVK5G2SAm5vtrv7KLKeOeYTxlZBkDRE6I7Toybt0A/viewform 11. Google - STEP (Student Training in Engineering Program) STEP (Student Training in Engineering Program), formerly known as Engineering Practicum, is a 12-week internship for first and second-year undergraduate students with a passion for computer science. 👉 https://buildyourfuture.withgoogle.com/programs/step 12. Igalia Coding Experience The Igalia Coding Experience is a grant program for people studying Computer Science, Information Technology, or Free Software, whether in a formal setting (for example, a college or university) or an informal one (self-teaching). 👉 https://www.igalia.com/coding-experience/ 13. CERN Summer Student Programme If you’re studying computer science, you can join the student program, to work on an advanced IT project and follow IT lectures specially prepared for you by experts at CERN and other institutes. 👉 https://careers.cern/summer 14. FOSSEE GIS Internship Semester-long internships and Summer fellowships are organized each year. These tasks can involve programming/ scientific computing/ conducting awareness workshops/ collecting data that will be of use to the community. 👉 https://fossee.in/fossee-internships 15. VSRP Internships This program has over 100 projects spanning 16 STEM disciplines, and each project has its own 'Apply' button. Each project is 3 - 6 months long and includes airfares, flights, housing, health insurance, and a monthly stipend. 👉 https://vsrp.kaust.edu.sa/internship/search 16. Free Software Foundation Internships and Traineeships Generally open for applications throughout the year. Sometimes they offer longer internships and traineeships, and other times they seek applicants with a specific focus (for instance students for summer internships). 👉 https://fsfe.org/about/jobs/internship 17. OGP Internship You will have a direct hand working on things that matter. There will be opportunities to learn, contribute, and have the autonomy to work on tech for the public good. 👉 https://www.open.gov.sg/careers/internships/ 18. SheCodeAfrica Programs Programs are specifically designed to empower young African girls and women in tech with the technical and career skills needed to scale in their fields. 👉 https://shecodeafrica.org/ 19. Layer5 Internship Program Interns engage as part of the community. Your contributions will affect people you've never met as the Layer5 projects are being broadly referenced and used in organizations large and small. 👉 https://layer5.io/careers/internships 20. Season of KDE Every year since 2013, KDE Student Programs has been running Season of KDE as a program similar to, but not quite the same as Google Summer of Code, offering an opportunity to everyone (not just students) to participate in both code and non-code projects that benefits the KDE ecosystem. 👉 https://season.kde.org/ 21. Summer of Bitcoin A global, online summer internship program focused on introducing university students to Bitcoin open-source development and design. 👉 https://www.summerofbitcoin.org/ Did you like the resources? Here's more 👇 Over the previous months, I've compiled The Ultimate Bundle with every resource you need from starting to learn to code to landing your dream tech role! Includes 900+ resources in total, further divided into 99 categories for easier navigation and access! Save 1000s of hours and focus on what matters! I'm currently offering 50%OFF just for my readers! Don't miss out on this! ⭐ ! ⭐ Get it here: Ultimate Resources Bundle to Land a Job in Tech This package includes: : A comprehensive collection of resources, including tutorials, articles, and handbooks, to help you master coding and programming languages. 150+ Learning Resources to Learn to Code : A huge list of courses offered by reputable companies and universities, covering a wide range of development topics to enhance your skills and knowledge. 220+ Developer Courses from Top Companies and Universities : Obtain valuable certificates in coding and programming, which can significantly enhance your resume and demonstrate your expertise to potential employers. 140+ Coding Certificates to Boost Your Resume : Explore a wide list of tech internships that offer real-world experience, allowing you to kickstart your career and build a strong foundation in the tech industry. 110+ Tech Internships to Gain Experience : A comprehensive guide featuring essential tips, strategies, and resources to help you prepare for and succeed in tech job interviews. 160+ Interview Essentials for Job Seekers in Tech : Discover a wide array of job boards and platforms tailored to tech professionals, making it easier to find the perfect job opportunity in the tech industry. 120+ Tech Job Boards to Find Your Dream Role ⭐ ! ⭐ Get it here: Ultimate Resources Bundle to Land a Job in Tech Writing has always been my passion and it gives me pleasure to help and inspire people. If you have any questions, feel free to reach out! Make sure to receive the best resources, tools, productivity tips, and career growth tips I discover by subscribing to ! my newsletter Also, connect with me on , , and ! Twitter LinkedIn GitHub Also published . here

Exploring Programming Internships  for Tech Experience

Technology is one of the most impactful drivers of transformation and innovation. As more people adopt different tech concepts, more proponents have called for increased collaboration across the board. Today, open-source technologies provide a way for people to connect and contribute to the growth of different products and services. With technology touching different facets of our existence and people looking to get on the forefront of this development, there is no doubt that open-source technologies are necessary for growth and diverse advancement. However, the tech landscape we live in is undergoing a major shift. Web3 is currently the biggest craze on the internet right now, describing a future where the internet becomes more privacy-enabled and user-controlled. Already, there are several platforms in the Web3 space that are looking to usher in the next evolution of the Internet. And, Tea is one service looking to help software contributors earn money while building the world of Web3. In this interview, we spoke to Max Howell - creator of Homebrew as well as the founder of Tea & pkgx - to understand his vision for the platform and its future. 1. How do you see the future of Web3 going forward? Web3 isn’t new. Web2 wasn’t new. They were both natural evolutions of the Internet. The Internet has always been built by the hackers. The nerds who fricking love the technology they are building and don’t care about anything else. In my 20s I spent every waking moment at my computer working on open source with randoms I met on the Internet who just happened to agree with me: we were building something cool that was going to make the world a better place. It turned out we built web2. The foundations of it anyway. Web2’s fortunes are entirely built on top of open source. Our free labor with no compensation. Most of us started to realize a few years ago that we’d helped build something awful. Centralizing control with people who took advantage of it and demonstrated how much they didn’t deserve it. Centralization has allowed the worst sides of humanity to emerge, even worse. The people who built web1 built web2 and are building web3. Its ascent is inevitable. We just have to correctly incentivize the builders. 2. Are you a proponent of big companies building Web3 platforms or individual developers? I am a huge believer in the open source development model. Despite attempts to tame it and control it by huge enterprise companies like Microsoft and Sun during the early 2000s it rose up to dominate our entire world in spite of the wealth they burnt to try and stop us. Open source is not corporate. Open source is not capitalist. Open source is something else. This is why it has come to exist in all software at every layer of the stack. Every layer but the very top, the 1% that is proprietary code that is how web2 has made their fortunes without paying for 99% of their codebases. Open Source is something else. Which is why it has taken something new to figure out how to make open source sustainable. That something else is smart contracts on the tea protocol. Nothing else will work. 3. What do you think is the biggest factor in Web3 adoption? Stop talking about web3. Stop talking about blockchain. Stop talking about decentralization. Normal people are not swayed by this shit, by buzzwords. They care about features, about the value they can extract from products. Talk about why your product is better than anything web2 can make. Only talk about the technology with technologists. 4. How does Tea’s reward system work? The existing systems suffer from what we call the “favorites” problem. There’s 50 million open source projects. A fraction of a percent of them are popular enough, let alone even known well enough, to receive bounties or sponsorship. We see this emerge on a regular basis where fundamental pieces of open source encounter major issues which have an exponential effect on the entire software ecosystem. tea sees how the problem with fixing open source comes from the very nature of open-source which is entirely different from any other systems humanity has built. The open source model works because we have built amazing technologies to rapidly iterate. The entire software ecosystem includes a myriad of tools, some of which are called package managers. With these tools small contributions can be used as a foundation to huge achievements, like bricks laid in a tower. Because of this, a typical app can be composed of tens of thousands of open source projects. There’s no way to reward all these projects without understanding all the connections between them and distributing rewards based on that ranking. tea’s oracle system crunches the numbers using information from all the different package managers to determine the impact of each individual project to the entire open-source ecosystem. We call it Proof of Contribution. Registered projects earn rewards based on their impact and anyone in the community can stake tokens into that system to demonstrate their support for the project, improve its reputation and, in return, earn epoch rewards. This approach and its associated penalties when issues are reported but ignored, contribute to the security of the software supply-chain, so to reward those who reviewed the code and put their stake behind it, those rewards are passed to both the staked projects, and the staker, as thanks for securing the software supply-chain. Also, the rewards that pass to staked projects are split and passed to that project’s dependencies, so on and so forth to ensure that all contributions in the tree are rewarded for their efforts. 5. Would you prefer a developer-led Web3 frontier or do you believe that collaboration still plays a key role? Developers are the pioneers. But we are building technology for people and thus we must bring other kinds of people in. As I said before, web3 is still in the “whatever” phase for most people. Many don’t remember but web2 was also scorned at its infancy. What changed was that people learned how to market their use of web2 technologies properly. They achieved this by starting to bring other kinds of people into the fold. Web3 is devs and finance people RN. We need to bring in the next genre of people. 6. How does tea protocol incorporate the TEA token into its operating model? TEA is a cryptographic token which serves as the access key to certain parts and designated features of the tea Protocol. Holders of TEA token have the ability to: Register their packages; Support packages by staking TEA tokens to specific packages; Contribute to the security of the software supply chain by challenging packages and conducting reviews to report bugs and/or vulnerabilities. The tea protocol unlocks the open-source economy and creates value for builders, maintainers, and end-users of enterprise software. Ultimately, the value captured by the tea protocol accrues back to token holders as the community scales, creating a feedback loop that further incentivizes participation. Conclusion: With the tech landscape witnessing a seismic shift, it’s understandable that many are concerned about how collaboration will work going forward. And as Web3 takes shape, platforms like Tea are here to ensure that developers get their due.

ODEN TECHNOLOGIES

oden.io

-- employees

Since n.d.

COMPANY RANKING

ODEN TECHNOLOGIES

EVERGREEN INDEX #7395

Representing this company? Click here to claim and customize this page!

Oden Technologies WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

ODEN TECHNOLOGIES

oden.io

-- employees

Since n.d.

COMPANY RANKING

ODEN TECHNOLOGIES

EVERGREEN INDEX #7395

RELATED COMPANIESRANK

STORIES ABOUT ODEN TECHNOLOGIES

Representing this company? Click here to claim and customize this page!

Oden Technologies WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!