Get the most recent info and news about DataXpie on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

DataXpie is a Digital Content and Service Selling Platform. We are a team of experienced workers from all over the world. We provide a wide range of digital content and services with a complete solution for your specific needs. We offer a complete solution, from beginning to end. From concept to reality.

DataXpie

dataxpie

It seems like your message got cut off and you didn't specify what you're looking for. However, if you're looking for a list of web development technologies, I can certainly provide that ...

Web development technologies list

HackerNoon Reporter: Please tell us briefly about your background. I ( ) am a technology inventor by heart — I build startups. I have been a Bangladeshi entrepreneur for 21 years. I was born on August 09, 1999, in Sunamganj, Bangladesh. My father is a businessman and my mother is a housewife. Estiuck Al Regun Around age 11, I developed an interest in web designing and DIY electrical projects. I learned web designing through reading blogs and ebooks, and by age 12, I sold the design of a website I created on the internet for approximately $100. Besides that, I have a deep understanding of cybersecurity to monitor and fight threats to an organization’s IT infrastructure and assess security systems and measure for weaknesses and possible improvements. What's your startup called? And in a sentence or two, what does it do? is a Digital Content Selling Platform where a buyer can buy any digital content or services and anyone can sell their digital content according to their dream price. We founded the company on 1 December 2018. DataXpie Our mission is to grow and cultivate Bangladesh’s largest digital marketplace, a place where people can find and purchase any digital content and service they need, and build any business they dream of. What is the origin story? 5 years ago 21.44 Million people were connected to the internet but right now it’s 3 times bigger and in July 2021 according to the report of BTRC it was 112.719 Million. The price of Software and Graphic Design is still so expensive. So we are working like a two-sided platform where an author can sell his content or service and a buyer can buy his content or service. Our goal is to ensure low-cost better digital content and services. As a result, the usage of technology will be increased in developing countries. What do you love about your team, and why are you the ones to solve this problem? I’ve been lucky to find myself a part of the most incredible team, we’re now at 10+ people globally and it feels like the best mix of skillsets and diverse perspectives I could have hoped to create. Scaling fast meant some incorrect hires, but everyone at DataXpie is still working happily on a team today. I believe A dream doesn't become reality through magic; it takes sweat, determination and hard work and solving problems is my dream. If you weren’t building your startup, what would you be doing? Probably working as an IT officer for any company. Back in 2018, I was offered jobs at different IT companies and institutes but a line that changed my thoughts about life is Aspire to Inspire before you Expire At the moment, how do you measure success? What are your core metrics? Making an impact on human life and letting people’s dream come true through our startup is our main metric of success. For that, we have received many awards, such as: The Bizz Awards 2020 GP Accelerator 2.0 BASIS National ICT Award 2018 Champion at Robolution 2019 What’s most exciting about your traction to date? We have increased GMV YOY 2019-2020 by 300%. YOY 2020-2021 by 400% to date. What technologies are you currently most excited about, and most worried about? And why? I’m most excited about quantum computers for security and encryption and deep learning technology. Besides that, I am also worried about those things 😂. What advice would you give to the 18-year-old version of yourself? Keep learning. Know how to research and make that a habit. Change your friend circle if they don’t know about quantum computers, deep learning, and passive earning. What advice would you give to 18-year-old Bangladeshi teenagers? Especially for 18-year-old Bangladeshi teenagers don’t waste your time by gossiping. We are the hope of our family and country. It’s high time to grow your thoughts. Stop complaining about your country. Don’t ask what your country can do for you? Ask yourself what you can do for your country? Because we are the future of this country ❤️ Don’t forget to make your own story and keep learning through smiling 😉. Vote for DataXpie as the startup of the year, Sylhet

Startup Interview with Estiuck Al Regun, Founder of DataXpie

138 Stories To Learn About Startups Top Story

240 Stories To Learn About Startups Of The Year

356 Stories To Learn About Ecommerce

In 2021, hosted the first-ever awards!! HackerNoon Startup of the Year The goal of this campaign was to give recognition to the hardworking startups who pulled through the pandemic -- growing and thriving (despite all of the challenges). More than 44,000 startups were nominated all across the globe! They were selected from over 4,553 cities, with a total number of * cough * “4 billion” votes. Insane, right? To see who is a winner in each respective city, visit and pick a city. The name at the top of the list is the winner. (Compromised awards are under review. Cities with tie votes will be updated to have two winners!) startups.hackernoon.com The will announce the winners per region in their respective cities. Startups Winners Announcement series Here are the winners in the : Asia region Top Startup Cities in East Asia Seoul The Startup of the Year in is with 377 votes! Seoul dable Central is the Startup of the Year! It came out ahead with 121 votes. UpNow Central Sheung Wan came out as the Startup of the Year in with 154 votes. Votee Sheung Wan Tokyo The Startup of the Year in is with 41 votes! Tokyo ACES Wan Chai The Startup of the Year in is with 54 votes! Wan Chai Zetl Osaka, Japan In , is the winner! Osaka Nanophoton Taiwan came out as the Startup of the Year in ! Tico Taiwan Dongcheng, China In , is declared the Startup of the Year! Dongcheng Waiju Taipei came out as the Startup of the Year in ! Tico Taipei Akasaka In , is the winner! Akasaka WAKUWAKU Chuo In , is declared the Startup of the Year! Chuo GO-NET Ginza came out as the Startup of the Year in ! Netyear Ginza Kanazawa In , is the winner! Kanazawa Granzella Minami-aoyama In , is declared the Startup of the Year! Minami-aoyama Tokyo6 Entertainment Shibuya came out as the Startup of the Year in ! Hupro Japan Shibuya Toranomon In , is the winner! Toranomon Small Worlds Toshima In , is declared the Startup of the Year! Toshima Altplus Guangzhou, China came out as the Startup of the Year in ! Leyaoyao Guangzhou Hong Kong, Hong Kong In , is declared the Startup of the Year! Hong Kong Episode Six Kunming In , is the winner! Kunming Green A Sapporo In , is declared the Startup of the Year! Sapporo Polar Star Space Shirokanedai came out as the Startup of the Year in ! Nupp1 Shirokanedai Wuhan In , is the winner! Wuhan CSICEE Baoan, China In , is declared the Startup of the Year! Baoan Jumper Baoshan, China In , is declared the Startup of the Year! Baoshan Renren Games Chengdu, China came out as the Startup of the Year in ! Ju You Xi Chengdu Fujisawa, Japan In , is the winner! Fujisawa Scohia Pharma Fukuoka, Japan In , is declared the Startup of the Year! Fukuoka Kyulux Haidian, China came out as the Startup of the Year in ! 51vv Haidian Hongkou, China In , is the winner! Hongkou Kidstone Jiading, China In , is declared the Startup of the Year! Jiading IMRobotic Kawasaki, Japan came out as the Startup of the Year in ! QD Laser, Inc. Kawasaki Kyoto, Japan In , is declared the Startup of the Year! Kyoto Hacarus Minato, Japan In , is the winner! Minato Visionary Fun Nagoya, Japan came out as the Startup of the Year in ! Prodrone Nagoya Ningbo, China The winner in is . Ningbo OlymTech Corporation Aomori In , is declared the Startup of the Year! Aomori Forte Japan Bandai-cho In , is the winner. Bandai-cho Delta-Fly Pharm Changchun In , the Startup of the Year is . Changchun Kaidi Animation Fuchu In , is declared the Startup of the Year! Fuchu TMS Fukuoka-shi The winner in is . Fukuoka-shi SkyDisc Fukushima In , the Startup of the Year is . Fukushima Matsuri Technologies Gotanda In , is the winner. Gotanda Basset Hachioji In , is declared the Startup of the Year! Hachioji Adriakaim Harbin The winner in is J . Harbin oining Technology Iwaki In , is the winner. Iwaki Tao Engineering Izumo In , the Startup of the Year is . Izumo PuREC Kagoshima In , is declared the Startup of the Year! Kagoshima Farmers Support Kamakura In , the Startup of the Year is . Kamakura KAYAC Kami-meguro The winner in is . Kami-meguro Buzzreach Kanagawa In , is the winner. Kanagawa LoiLo Kashiwa In , the Startup of the Year is . Kashiwa Intelligent Surfaces Kita In , is declared the Startup of the Year! Kita Ax Robotix Kobe The winner in is . Kobe Coconomi Koriyama In , the Startup of the Year is . Koriyama Tocoton Kumamoto In , is the winner. Kumamoto Bloom Technology Kunitachi In , is declared the Startup of the Year! Kunitachi Trys Liaocheng The winner in is . Liaocheng Jinzheng Animation Minami In , is declared the Startup of the Year! Minami ATLAB Miyazaki In , is the winner. Miyazaki Caster Nagano In , is declared the Startup of the Year! Nagano Precision Forestry Nagaoka The winner in is . Nagaoka Plant Form Nihombashi-hamacho In , the Startup of the Year is . Nihombashi-hamacho Shizen Energy Niigata In , is declared the Startup of the Year! Niigata UniBio Japan Nishi-shinagawa In , is the winner. Nishi-shinagawa Play Heart Okayama In , is the winner! Okayama Arch Farm Okinawa In , is declared the Startup of the Year! Okinawa BlogNinja Otemachi In , the Startup of the Year is . Otemachi beBit, Inc Qingdao Shi In , is the winner! Qingdao Shi Manyoubang Quanzhou In , is declared the Startup of the Year! Quanzhou KungFu Animation Group Roppongi The winner in is . Roppongi Holo-X Saitama In , the Startup of the Year is . Saitama GameTector Sendai In , is the winner! Sendai Sendai Smart Machines Setagaya In , is declared the Startup of the Year! Setagaya Skyestate Shenzhen In , the Startup of the Year is . Shenzhen Aiator Shinjuku In , is declared the Startup of the Year! Shinjuku Scalar Shizuoka In , is the winner! Shizuoka PiPhotonics Sumida The winner in is . Sumida Holotch Tachikawa In , the Startup of the Year is . Tachikawa PlusOne Takamatsu In , is declared the Startup of the Year! Takamatsu Ken Robotec Tokushima In , is the winner! Tokushima DennoKotsu Toyonaka In , the Startup of the Year is . Toyonaka FISH BIOTECH Tsukuba In , is declared the Startup of the Year! Tsukuba WARPSPACE Wakayama In , the Startup of the Year is . Wakayama 4D Sensor Xiangzhou In , is the winner! Xiangzhou ALLiN Edutech Group Xianyang The winner in is . Xianyang TONLY Xiaoshan In , is declared the Startup of the Year! Xiaoshan GONGHUI Yamaguchi In , the Startup of the Year is . Yamaguchi Archis Yangpu The winner in is . Yangpu Tmxmall Yau Tsim Mong In , is the winner! Yau Tsim Mong Airnugget Yeoksamdong In , the Startup of the Year is . Yeoksamdong WizSchool Yichang In , is declared the Startup of the Year! Yichang Hubei Kento Electronic Yokohama In , is the winner! Yokohama Next Innovation Yubei In , the Startup of the Year is . Yubei Ophiux Network Yueqing The winner in is . Yueqing Yueqing Easythink Media Zhuhai In , is the winner! Zhuhai Aijuba Hongo In , the Startup of the Year is . Hongo Smile Robotics, Inc Admiralty In , is declared the Startup of the Year! Admiralty BUNCH Southern In , the Startup of the Year is . Southern Retykle Yongdungpo In , is the winner! Yongdungpo Push News Taichung In , is declared the Startup of the Year! Taichung Wondercise Cyberport, Hong Kong In , the Startup of the Year is . Cyberport Animoca Brands Hefei, China In , is the winner! Hefei Tsinglink New Territories, Hong Kong The winner in is . New Territories, Hong Kong Zeek Dezhou In , the Startup of the Year is . Dezhou Dezhou Baji Kitakyushu In , is declared the Startup of the Year! Kitakyushu Reif Seongnam In , is the winner! Seongnam EOFlow Yanshi In , the Startup of the Year is . Yanshi CASI Vision Top Startup Cities in South Asia Hyderabad, India The Startup of the Year in is with 962 votes Hyderabad Quixy Lahore, Pakistan is the Startup of the Year! It came out ahead with 186 votes. SmartWindows Lahore Bangalore, India When it comes to , is the Startup of the Year with 210 votes! Bangalore vBridge Hub Pune, India couldn’t be beaten for Startup of the Year in , it won with 129 votes! InfraCloud Pune Jaipur, India The Startup of the Year in is with 166 votes Jaipur Emizen Tech Lucknow, India is the Startup of the Year! It came out ahead with 279 votes. Keeros Superfoods Lucknow Noida, India came out as the Startup of the Year in with 156 votes! Bagisto Noida Mumbai, India When it comes to , is the Startup of the Year with 96 votes! Mumbai Sequretek Sylhet couldn’t be beaten for Startup of the Year in , it won with 122 votes! DataXpie Sylhet Mylapore, India In , is the winner! Mylapore Yalimart Goa, India In , the Startup of the Year is Goa Filter Coffee Delhi, India In , is declared the Startup of the Year! Delhi EkAnek Networks Karachi, Pakistan The winner in is Karachi Bykea Ahmedabad, India In , is the winner Ahmedabad MageComp Dhaka, Bangladesh The winner in is Dhaka SOLshare Islamabad, Pakistan In Islamabad, is declared the Startup of the Year! TAG Gurgaon, India In , the Startup of the Year is Gurgaon ixigo Bengaluru, India In , is the winner! Bengaluru Prakshep Private Limited Haryana, India couldn’t be beaten for Startup of the Year in ! Footprints Childcare Haryana Kolkata, India In , is the winner! Kolkata Wow! Momo New Delhi, India In , is declared the Startup of the Year! New Delhi Rusk Media Indore, India The winner in is Indore Chattar Patar Kormangala In , is the winner Kormangala Sanctum Technologies Ghaziabad came out as the Startup of the Year in Craftfurnish.com Ghaziabad! Mohali In , the Startup of the Year is Mohali DataKund Mysore The Startup of the Year in is Mysore Excelsoft Karnal came out as the Startup of the Year in ! Dunar Karnal Kolhapur In , is the winner! Kolhapur Zanvar Group Patiya In , the Startup of the Year is Patiya Kloud Project Kathmandu In , is declared the Startup of the Year! Kathmandu Vairav Technology Bangalore City, India In is the winner! Bangalore City, PagarBook Ernakulam The winner in is Ernakulam Epixel Solutions Colombo, Sri Lanka couldn’t be beaten for Startup in the Year in MAs Foods Colombo Andheri, India When it comes to , is the Startup of the Year! Andheri DoorMint Kanpur, India came out as the Startup of the Year in ! ServiceWalaa Kanpur Faisalabad In , is the winner! Faisalabad DigiKhata Nasik The Startup of the Year in is Nasik Sahyadri Farms Patna came out as the Startup of the Year in ! Asvin Tech Patna Chennai, India In , is the winner! Chennai Agnikul Cosmos Faridabad, India In , is declared the Startup of the Year! Faridabad UClean Panjim, India The winner in is Panjim TempoGO Ranchi, India In , is the winner Ranchi Osam Dairy Surat, India In , is declared the Startup of the Year! Surat Wishbook Thane, India In , is the winner! Thane Delight Foods Trivandrum, India came out as the Startup of the Year in ! GenRobotic Trivandrum Vijayawada, India In , the Startup of the Year is Vijayawada Eruvaka Technologies Visakhapatnam, India In , Ltd is declared the Startup of the Year! Visakhapatnam Greenline Labs Pvt Ahmadabad In , is the winner! Ahmadabad KiwiQA Services Bandra came out as the Startup of the Year in ! Car Club Bandra Bombay In , is the winner! Bombay Locobuddy Mobile Technology (Sportobuddy) Cochin In , the Startup of the Year is Cochin Tuttifrutti Interactive Gandhinagar In , is declared the Startup of the Year! Gandhinagar Kroop AI Guwahati In , the Startup of the Year is Guwahati GNRC Hospitals Jamshedpur In , is declared the Startup of the Year! Jamshedpur Intelliber Kabul The winner in is Kabul Miracle Shop (Vocinno Technology) Karur In , is the winner Karur Karur Vysya Bank Kochi The winner in is Kochi BestDoc Kozhikode In , is declared the Startup of the Year! Kozhikode OfficeKit HR Ludhiana In , the Startup of the Year is Ludhiana Trendy Beat Network Madhapur In , is the winner! Madhapur Head Digital Works Mangalore When it comes to , is the Startup of the Year Mangalore Hangyo Nagpur came out as the Startup of the Year in ! UPrudence Agro Nagpur Rawalpindi The Startup of the Year in is Rawalpindi Pro Options Marketing Rishikesh came out as the Startup of the Year in ! Descatuk Rishikesh Secunderabad couldn’t be beaten for Startup of the Year in Prakriti Biosystems Engineering Secunderabad Tirupur In , is the winner! Tirupur Tubeknit Fashions Udupi The winner in is Udupi Krishitantra Top Startup Cities in Southeast Asia Singapore, Singapore The Startup of the Year in is with 6556 votes! Singapore Cake DeFi Hanoi, Vietnam is the Startup of the Year! It came out ahead with 683 votes. DeFi For You Hanoi Kuala Lumpur, Malaysia came out as the Startup of the Year in with 109 votes! Trambellir Kuala Lumpur Pasig, Philippines When it comes to , . is the Startup of the Year with 110 votes! Pasig Quickreach Inc Bangkok, Thailand couldn’t be beaten for Startup in the Year in , it won with 60 votes! Perception Codes Bangkok Johor Bahru In , the Startup of the Year is . Johor Bahru AGR1 Bandung, Indonesia In , is declared the Startup of the Year! Bandung Prosa Portugal The winner in is . Portugal SCEMAI Tangerang, Indonesia In , is the winner Tangerang Yummy Corp Jakarta, Indonesia In , is the winner! Jakarta Tokocrypto Dki Jakarta, Indonesia In , is declared the Startup of the Year! Dki Jakarta Limakilo Bangkok Metropolis In , is the winner! Bangkok Meticuly Medan The winner in is ! Medan DIKA Manila, Philippines In , is declared the Startup of the Year! Manila PawnHero Phnom Penh, Cambodia In , the Startup of the Year is Phnom Penh vKirirom Bali In , is the winner! Bali Radius App Penang When it comes to , is the Startup of the Year Penang EasyParcel Surabaya came out as the Startup of the Year in ! Super Surabaya Setiabudi, Indonesia The Startup of the Year in is Setiabudi Chilibeli Cebu City came out as the Startup of the Year in MiCab Cebu City! Jawa couldn’t be beaten for Startup in the Year in DAUR Jawa Ho Chi Minh City, Vietnam In , is the winner! Ho Chi Minh City Pharmacity Selangor, Malaysia is the Startup of the Year! BabyDash Selangor Jakarta Pusat, Indonesia In , is the winner! Jakarta Pusat StickEarn Semarang When it comes to , is the Startup of the Year! Semarang Food Bear Petaling Jaya, Malaysia The winner in is . Petaling Jaya Deemples Kebayoran Baru, Indonesia couldn’t be beaten for Startup in the Year in ! Yogrt Kebayoran Baru Kuningan The winner in is . Kuningan Prinzio Bukit Merah came out as the Startup of the Year in ! DC Alliance Bukit Merah Huai Khwang In , the Startup of the Year is . Huai Khwang AppMan Co.,Ltd Kuta In , is declared the Startup of the Year! Kuta Gringgo Mandaluyong came out ahead as the Startup of the Year! Work Inspire Mandaluyong Midview City In , is the winner! Midview City BioMers Phuket The Startup of the Year in is ! Phuket Sapahn Yangon, Myanmar The winner in is . Yangon Wave Money Denpasar When it comes to , is the Startup of the Year! Denpasar Revasi Nakhon Ratchasima, Thailand The winner in is . Nakhon Ratchasima Siam Lottery co., Ltd Taguig, Philippines In , is the winner! Taguig PayMongo Bogor came out as the Startup of the Year in ! TIRANYX Bogor Senai, Malaysia won as the Startup of the Year. Nutrition Technologies Senai Bekasi Kota couldn’t be beaten for Startup in the Year in ! 8villages Bekasi Kota Top Startup Cities in Middle East Tel Aviv, Israel The Startup of the Year in is with 1,065 votes! Tel Aviv hyro Beirut, Lebanon is the Startup of the Year! It came out ahead with 1,276 votes. Hovi Digital Lab Beirut Riyadh, Saudi Arabia With 350 votes, came out as the Startup of the Year in ! Flicron Riyadh Tel Aviv-yafo, Israel When it comes to , is the Startup of the Year with 167 votes! Tel Aviv-yafo Cymbio Haifa, Israel couldn’t be beaten for Startup in the Year in , it won with 183 votes! Pico Haifa Amman, Jordan In , with 141 votes, is the winner! Amman Edexy Dubai, United Arab Emirates In , the Startup of the Year is with 74 votes. Dubai Ziina Doha, Qatar In , is declared the Startup of the Year! Doha SkipCash Tehran, Iran The winner in is . BeyondBoard AR Tehran Jerusalem, Israel In , is the winner! Jerusalem SolidBlock Rishon Le Zion The Startup of the Year in is ! Rishon Le Zion Passparto Ness Ziona, Israel is the Startup of the Year! Soapy Ness Ziona Kuwait, Kuwait came out as the Startup of the Year in ! Brapper Shop Kuwait Manama, Bahrain couldn’t be beaten for Startup in the Year in ! Doctori Manama Lod, Israel is the winner in ! Phinergy Lod Agudat Israel, Israel When it comes to is the Startup of the Year with [vote ] votes! Agudat Israel, Aqaurius Engines Beersheba, Israel In , the Startup of the Year is . Beersheba Visual Estate Ltd Giv'atayim, Israel In , is declared the Startup of the Year! Giv'atayim WSC Sports Modi'in, Israel The winner in is . Modi'in Loginno Nazareth, Israel In , is the winner! Nazareth AR Guide In Medical Ramat Gan, Israel The Startup of the Year in is . Ramat Gan Datos Health Baghdad When it comes to , is the Startup of the Year! Baghdad Mishwar Khobar In , the Startup of the Year is . Khobar Garage Misgav couldn’t be beaten for Startup in the Year in ! Eden Shield Misgav Sharjah came out as the Startup of the Year in ! Bookends Sharjah Affula In , the Startup of the Year is . Affula eMotion Technologies Bene Beraq, Israel In , is the winner Bene Beraq Reflectiz Caesarea, Israel In , the Startup of the Year is . Caesarea CardiacSense Dubayy In , is declared the Startup of the Year! Dubayy North Ladder Muscat couldn’t be beaten for Startup in the Year in ! iLab Marine Muscat Tirat Carmel In , the Startup of the Year is . Tirat Carmel PassCall Advanced Technologies Hod Hasharon, Israel The winner in is . Hod Hasharon Idomoo Holon, Israel came out ahead as the Startup of the Year! OptimalPlus Holon Jeddah, Saudi Arabia When it comes to , is the Startup of the Year! Jeddah Arabian Combustion Technologies Ramat Hasharon, Israel is the Startup of the Year! Lumen Ramat Hasharon Al Khobar came out as the Startup of the Year in ! Swyft Al Khobar Bet Shemesh In , is the winner! Bet Shemesh Degania Medical Binyamina When it comes to , is the Startup of the Year! Binyamina OncoHost Makkah In , the Startup of the Year is Makkah Scimagine Or Akiva Last but not least, couldn’t be beaten for Startup in the Year in the city of ! WMT Or Akiva More Information: If you’re a winner, please answer ! If you’re a reader, participant, just a curious individual, consider answering one of our interviews . this interview here Read the FAQs about or . Startups here here To see all of the winners you may visit , pick any city and view the name that appears at the top. this site If you’re interested in partnering with us, we invite you to look through some of these or reach out to our business department. unique sponsorship options Perhaps creating your very own tickles your fancy or maybe you have a whole new idea for how we can partner! writing contest Thank you to all of our sponsors! We truly appreciate your contribution to the tech world and thus the future! The Startups Awards are sponsored by: , , , , , , , , , , Blockster Brex FreeCodeCamp LaunchDarkly Coil Udacity Udemy Vercel Asia Pacific Transgender Network Bybit Everscale Network The post was written by Kushal Tamrakar and Valerie Ejegi, edited by Hang Ngo. If you have any questions or concerns or would like to report a discrepancy please email ellen@hackernoon.com.

Meet the Winners of Startups in Asia

Working With Wav2vec2 Part 1: Finetuning XLS-R for Automatic Speech Recognition

Wav2vec2 এর সাথে কাজ করা পার্ট 1: স্বয়ংক্রিয় বক্তৃতা স্বীকৃতির জন্য XLS-R ফাইনটিউনিং

Arbeiten mit Wav2vec2 Teil 1: Feinabstimmung von XLS-R für die automatische Spracherkennung

Trabajar con Wav2vec2 Parte 1: Ajuste de XLS-R para el reconocimiento automático de voz

Travailler avec Wav2vec2 Partie 1 : Affiner XLS-R pour la reconnaissance vocale automatique

Wav2vec2 के साथ कार्य करना भाग 1: स्वचालित वाक् पहचान के लिए XLS-R को परिष्कृत करना

Wav2vec2 の操作パート 1: 自動音声認識のための XLS-R の微調整

Wav2vec2 작업 1부: 자동 음성 인식을 위한 XLS-R 미세 조정

Trabalhando com Wav2vec2 Parte 1: Ajuste fino do XLS-R para reconhecimento automático de fala

Работа с Wav2vec2. Часть 1. Точная настройка XLS-R для автоматического распознавания речи

Wav2vec2 ile Çalışmak Bölüm 1: Otomatik Konuşma Tanıma için XLS-R'ye İnce Ayar Yapmak

Làm việc với Wav2vec2 Phần 1: Tinh chỉnh XLS-R để nhận dạng giọng nói tự động

使用 Wav2vec2 第 1 部分：微调 XLS-R 以实现自动语音识别

A few days ago, I underwent an MRI scan. They slid me into a large tube, and for 15 minutes, the machine around me buzzed, hummed, and clicked. At the end of the examination, I received a CD containing the data. What does a good developer do in such a situation? Of course, as soon as they get home, they start examining the data and thinking about how they might extract it using . Python The CD contained a bunch of DLLs, an EXE, and a few additional files for a Windows viewer, which are obviously useless on Linux. The crux was in a folder named DICOM. It held a series of extension-less files across eight folders. ChatGPT helped me figure out that is a standard format typically used for storing MRI, CT, and X-ray images. Essentially, it's a simple packaging format containing grayscale images along with some metadata. DICOM The allows you to easily export the files in PNG format: following code import os
import pydicom
from PIL import Image
import sys

def save_image(pixel_array, output_path):
    if pixel_array.dtype != 'uint8':
        pixel_array = ((pixel_array - pixel_array.min()) / (pixel_array.max() - pixel_array.min()) * 255).astype('uint8')

    img = Image.fromarray(pixel_array)
    img.save(output_path)
    print(f"Saved image to {output_path}")

def process_dicom_directory(dicomdir_path, save_directory):
    if not os.path.exists(save_directory):
        os.makedirs(save_directory)

    dicomdir = pydicom.dcmread(dicomdir_path)

    for patient_record in dicomdir.patient_records:
        if 'PatientName' in patient_record:
            print(f"Patient: {patient_record.PatientName}")

        for study_record in patient_record.children:
            if 'StudyDate' in study_record:
                print(f"Study: {study_record.StudyDate}")

            for series_record in study_record.children:
                for image_record in series_record.children:
                    if 'ReferencedFileID' in image_record:
                        file_id = list(map(str, image_record.ReferencedFileID))
                        file_id_path = os.path.join(*file_id)
                        print(f"Image: {file_id_path}")

                        image_path = os.path.join(os.path.dirname(dicomdir_path), file_id_path)
                        
                        modified_filename = '_'.join(file_id) + '.png'
                        image_save_path = os.path.join(save_directory, modified_filename)

                        try:
                            img_data = pydicom.dcmread(image_path)
                            print("DICOM image loaded successfully.")
                            save_image(img_data.pixel_array, image_save_path)
                        except FileNotFoundError as e:
                            print(f"Failed to read DICOM file at {image_path}: {e}")
                        except Exception as e:
                            print(f"An error occurred: {e}")

if __name__ == "__main__":
    if len(sys.argv) < 3:
        print("Usage: python dicomsave.py <path_to_DICOMDIR> <path_to_save_images>")
        sys.exit(1)
    
    dicomdir_path = sys.argv[1]
    save_directory = sys.argv[2]
    process_dicom_directory(dicomdir_path, save_directory) The code expects two parameters. The first is the path to the DICOMDIR file, and the second is a directory where the images will be saved in PNG format. To read the DICOM files, we use the library. The structure is loaded using the function, from which metadata (such as the patient's name) and studies containing the images can be extracted. The image data can also be read with . The raw data are accessible through the field, which the function converts to PNG. It's not too complicated. Pydicom pydicom.dcmread dcmread pixelarray save_image The result is a few squares (typically 512x512) grayscale images, which can be easily utilized later on. Why did I think it was important to write this short story about my adventure using the DICOM format? When a programmer hears about processing medical data, they might think it's something serious, something that only universities and research institutes can handle (at least, that's what I thought). As you can see, we're talking about simple grayscale images, which are very easy to process and ideal for things like neural network processing. Consider the possibilities this opens up. For something like a simple tumor detector, all you need is a relatively straightforward convolutional network and a sufficient number of samples. In reality, the task isn't much different from recognizing dogs or cats in images. To illustrate how true this is, let me provide an example. After a brief search on Kaggle, where brain tumors are classified based on MRI data (grayscale images). The dataset categorizes the images into four groups: three types of brain tumors and a fourth category with images of healthy brains. The architecture of the network is as follows: I found a notebook ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                 │ (None, 164, 164, 64)   │         1,664 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d (MaxPooling2D)    │ (None, 54, 54, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 (Conv2D)               │ (None, 50, 50, 64)     │       102,464 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_1 (MaxPooling2D)  │ (None, 16, 16, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_2 (Conv2D)               │ (None, 13, 13, 128)    │       131,200 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_2 (MaxPooling2D)  │ (None, 6, 6, 128)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_3 (Conv2D)               │ (None, 3, 3, 128)      │       262,272 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_3 (MaxPooling2D)  │ (None, 1, 1, 128)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten (Flatten)               │ (None, 128)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 512)            │        66,048 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 4)              │         2,052 │
└─────────────────────────────────┴────────────────────────┴───────────────┘ As you can see, the first few layers consist of a convolution and max pooling sandwich, which extracts patterns of tumors, followed by two dense layers that perform the classification. This is exactly the same architecture used in that uses the CIFAR-10 dataset. I remember it well, as it was one of the first neural networks I encountered. The dataset contains 60,000 images categorized into ten classes, including two classes for dogs and cats, as I mentioned earlier. The architecture of the network looks like this: TensorFlow's CNN sample code CIFAR-10 Model: “sequential”
_________________________________________________________________
Layer (type) Output Shape Param # 
=================================================================
conv2d (Conv2D) (None, 30, 30, 32) 896 
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 15, 15, 32) 0 
_________________________________________________________________
conv2d_1 (Conv2D) (None, 13, 13, 64) 18496 
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 6, 64) 0 
_________________________________________________________________
conv2d_2 (Conv2D) (None, 4, 4, 64) 36928 
_________________________________________________________________
flatten (Flatten) (None, 1024) 0 
_________________________________________________________________
dense (Dense) (None, 64) 65600 
_________________________________________________________________
dense_1 (Dense) (None, 10) 650 
=================================================================
Total params: 122,570
Trainable params: 122,570
Non-trainable params: 0
_________________________________________________________________ It's clear that we are dealing with the same architecture, just with slightly different parameters, so recognizing brain tumors in MRI images is indeed not much more difficult than identifying cats or dogs in pictures. Furthermore, the mentioned Kaggle notebook is capable of classifying tumors with 99% effectiveness! I just want to point out how simple it can be to extract and process medical data in certain cases and how problems that seem complex, like detecting brain tumors, can often be quite effectively addressed with simple, traditional solutions. Therefore, I would encourage everyone not to shy away from medical data. As you can see, we are dealing with relatively simple formats, and using relatively simple, well-established technologies (convolutional networks, vision transformers, etc.), a significant impact can be achieved here. If you join an open-source healthcare project in your spare time just as a hobby and can make even a slight improvement to the neural network used, you could potentially save lives!

Can You Open Medical Data (MR, CT, X-Ray) in Python and Find Tumors With AI?! Maybe

আপনি কি পাইথনে মেডিকেল ডেটা (এমআর, সিটি, এক্স-রে) খুলতে পারেন এবং এআই দিয়ে টিউমার খুঁজে পেতে পারেন?! হতে পারে

Können Sie medizinische Daten (MR, CT, Röntgen) in Python öffnen und Tumore mit KI finden?! Vielleicht

¿Se pueden abrir datos médicos (RM, tomografía computarizada, rayos X) en Python y encontrar tumores con IA? Tal vez

Pouvez-vous ouvrir des données médicales (IRM, CT, rayons X) en Python et rechercher des tumeurs avec l'IA ?! Peut être

क्या आप पाइथन में मेडिकल डेटा (एमआर, सीटी, एक्स-रे) खोल सकते हैं और एआई के साथ ट्यूमर ढूंढ सकते हैं?! शायद

Pythonで医療データ（MR、CT、X線）を開き、AIで腫瘍を見つけることができるか？！ おそらく

의료 데이터(MR, CT, X-Ray)를 Python으로 열고 AI로 종양을 찾을 수 있습니까?! 아마도

Você pode abrir dados médicos (ressonância magnética, tomografia computadorizada, raio-X) em Python e encontrar tumores com IA?! Talvez

Можете ли вы открыть медицинские данные (МР, КТ, рентген) на Python и найти опухоли с помощью ИИ?! Может быть

Python'da Tıbbi Verileri (MR, CT, X-Ray) Açabilir ve Yapay Zekayla Tümörleri Bulabilir misiniz?! Belki

Bạn có thể mở dữ liệu y tế (MR, CT, X-Ray) bằng Python và tìm khối u bằng AI không?! Có lẽ

你能用 Python 打开医疗数据（MR、CT、X 射线）并用 AI 查找肿瘤吗？！也许吧

Introducing Our New JavaScript Standard Library

Businesses across the genres craving for , has been all about consistent revamping and digital evolution chains. Traversing from mainstream processes to tech-efficient landscapes has been a mammoth move and of course an imperative one. However, now one can say that scratching off the surface of digital transformation for businesses has come full circle with AI in metaverse. Where personalization, massive data optimization and immersive roll outs are easier than ever. But again, this is only the businesses capitalize on futuristic catalysts like ‘Fueling Innovation & Step-Ups’ if and when AI and Metaverse. Yes, there are virtual shopping, interactive displays, and AI-powered virtual assistants for sales. Or even the AI-powered virtual meeting platforms have been super tech-savvy catch-ups. But I can now safely say that it was quite last season because synergizing on the symphony of Metaverse with AI is the next wave. So, for all the modern businesses to hover around the ultra-competitive times and level up comprehensively on all fronts, calling dibs on Metaverse with AI is the only way out. The duo is not just rolling out benefits and perks in abundance for the businesses. They are changing the entire trajectory of digital transformation. So, let’s unravel the prowess of this tech-fusion ahead! The Need of the Hour: AI and Metaverse . But wait, is it the only catch when times have changed dramatically, and businesses are on an outlook for innovative leverages? Certainly, no! Right? AI and Metaverse have moved past the Traditional Bandwidths! Transcending limitations, Automating tasks and zapping the gap between the real and virtual worlds still does shine and serve for the businesses I mean, the swaps from manual to automation and clutching in the meeting rooms to highly hyper-realistic platforms are game changers. But, again would it streamline operations, as we live in the datasphere. Clearly businesses across the globe are equipped with data flowing from all directions and they do rely on it extensively. Well, was that it? Was that the big final? Without a second thought, NO! Okay, automation was magical. But considering the present times and the competitive march towards the future, integrating AI and the metaverse is not just a looming luxury but a necessity. Now, it’s natural to think why and how. For that, let us get you navigating through the same. Considering the cluster of futuristic capabilities that AI hosts, it is understood that the initial drive for the businesses to integrate AI was to add a ‘Dollop of automation, smart and modern dynamics’. However, with the digital horizons taking up on a massive scale, AI delivers more than ever. Why AI? Let’s explore your businesses as a ‘ Robust Machine.’ And for that machine to run efficiently, AI is the ‘Core Mechanism.’ It means if you, as a business loses the grip on mechanics, the machinery stops working. To be precise, it falls out completely if you let go of AI in digital transformation. For Example- Now, at this point it is quite natural of you to raise the most obvious question; Sure, but what does metaverse bring to the table? Why Metaverse? When it comes to businesses and considering it as a robust machine (for example, same as above), artificial intelligence is clustered machinery that is in charge of the process, production, and operational efficiency. Basically, it is a on the other hand, is a ‘Finished Product, which is not your average batch of products but a personalized, futuristic, and immersive one more like, . In other words, AI in metaverse are like two peas in a pod. And you cannot stack and put chips on one and leave the other. They just synergize so well together that no tech giant would want to ignore their massive impact. back-end ecosystem. , Metaverse a front end ecosystem Having said that- it does not matter if you are into retail, corporate, fintech or even narrowed down to marketing, production or designing. This ultimately brings us down to the HOW. How is this the only seed for taking ‘The next digital transformation leap’? And, for the same we need to tap onto the tech-trail. TECH-DUO How Integrating Metaverse and AI in digital transformation is the only move ahead for businesses? To help you understand it better, let us give you a simple formula. Technically Speaking- AI x Metaverse  (Backend + Frontend) Catering Modern-Day Business Requirements = Intelligent Agents of Computer Science fetching, formatting, filtering, and powering up data. AI: An Immersive space powered by AI, panning out real-life-like aspects for the users. Metaverse: In a nutshell, it does not matter what stream or genre your business falls into. The digital horizon of the times, roots on two thriving technologies i.e Metaverse and Artificial Intelligence. Right from operational efficiency, to acing a competitive edge in the market to delivering next-gen experiences to the users. This dynamic duo is certainly a ‘ solution, and integrating it is the only step to soar high for the businesses. one-size-fits-all’ Besides that, I do have some claims to make for AI in Metaverse; AI and Metaverse are . beyond the basics You can count on it for the ‘Innovative Next-Gen Dive.’ Yes, you ‘ Save money while facelifting your operations.’ No, you ‘Do not have to undertake a trail of tasks.’ Don’t worry, with below listed genres I will affirm the above claims. So, tag along! Training Simulations Break-free the mainstream physical conference room training which aren’t just passive but highly time and cost intensive. Metaverse digital transformation is here with immersive safety training with AI expertise to fuel in the adaptive, tailored yet cost effective training! Fusing in realistic scenarios in a spatial space with life-like learning sessions is what one can say that offers. But, I am still adamant about our claims. These training programs are cost-savvy, highly skill-efficient, and industry-agnostic. You can have medical professionals or even construction personnel to train. Metaverse with AI is the step-up you need. And when I say you need it, I mean it! Metaverse x AI Earlier the training sessions involved manual physical explorations, in-person training and direct machine/equipment contact, which was time consuming but at the same time extremely risky and even resource intensive. And, above all that, it’s now SO LAST SEASON! Here, I will tell you what gap it bridges. You are panning out collaborative virtual training with an expert who takes on a spatial space with all the required equipment/machinery available. By hopping onto this futuristic fusion, you are not just offering safety training and shrinking down the incidental risks. But, you are offering a simulated controlled environment, connecting through experts and dodging barriers. On top of that, you are cutting machine maintenance and operational costs. At the same time, it offers personalized, active, and real-time learning. Consider this: Transform Sales and Marketing Strategies If you thought AI powered chatbots, automated marketing campaigns and collaborative virtual meetings are all that AI and metaverse has gotten, then let me tell you. You are not just highly mistaken, but you are undermining its potential to a great extent. I will equip you with how Metaverse x AI can do wonders and revolutionize your sales and marketing. From consumer & company interaction to consumer x product interaction virtually. Brace Yourself: I meant for the fact all I know that the world is shrinking and businesses are trying their level best to modernize and level up digitally. To help you understand, let’s approach your brain wires in a simpler way. Earlier businesses used to analyze consumer preferences and market trends manually and showcased the products/services in mainstream displays. Personalization was a farfetched pick for businesses because coping with inconsistent consumer trends in the masses was not only difficult but impossible. There was a gap between consumers and businesses offerings due to lack of data analyzation, preference consideration and lack of customer engagement. For Sales: Be it campaigns, personalization virtual launches and down to chatbots everything is streamlined and up to the trends. Businesses can take a backseat in this digital age considering their putting the chips on this dynamic duo. However, this might not do justice to the claims stated above, which is why I’ll just quickly run you through the evolved sales process. With AI and Metaverse in Sales: How does it transform your Sales and Marketing processes? Pools in the data comprehensively, yet segregates the chunks into ‘Preferences, Patterns, Purchase history’ by leveraging predictive analytics. Further, the same data is brought to use in personalizing the entire marketing strategy and virtual immersive 3D launches. With that, AI seeps this data into the Virtual assistants in real-time thereby rolling out a personalized output to the users according to their preferences, likings and choices. AI – The entire data processing across the platforms and servers is lastly brought to show on the business front. Now this can be in the form of modern virtual product launches, 3D virtual try-ons, 3D product demonstrations and all of this in a hyper-realistic space i.e Metaverse. Metaverse Digital Transformation: Imagine virtually trying on a dress or getting to access a virtual test drive for that car that’s recently launched. It is now possible from your own living room but with all the immersiveness of actuality. Plus, what adds a cherry to the cake is all of this will be (automatically and in real-time) tailored to the customer’s preferences via virtual reality development. Not just that, you as a business can get vast amounts of data optimized (in no-time), display products in 3D space and let the products interact with your potential segment of customers (in real-time). And, and, and hit that sales target while finessing the modern marketing and sales game. A win-win right there like never before! Work and Play: Perfectly Balanced Gear up to bail out from the monotonous workloads! It’s time to hunch down the hamster wheel! Taking a backseat and letting things run on auto-pilot has always been synonymous with AI. But, now by syncing on the symphony of metaverse with AI it is no less than living in the future without waiting for 2050. The avatar aura is real and how. You are serving your clients round the clock and bringing in business globally. But there is no employee burnout issue nagging you, nor are the productivity dips hitting your sales anymore. Picture this: If you are running an e-commerce site- by counting on this duo. The AI powered chatbots do not just respond to the user-queries in real time. But, your users will be equipped with a 3D product display and that too tailored to their specific needs. All this is via virtual reality development. For Example- Your Perks: Customer attrition rate, offer modern services while saving resource costs. How time consuming and labor intensive is to run through numerous resumes, shortlisting potential candidates and further dealing with the onboarding process. But, it can be swapped to a rapid, streamlined, cost and time-savvy one. Every company understands the basics, if your employees are happy, they tend to perform better, and your company stays in business lane. Hence, what if there were virtual assistants powered by AI guiding your interns to understand the basics and recognize their potential? I mean think how quickly you will be able to streamline your HR processes before and after onboarding of your employees. Now Picture This- Implementing Metaverse with AI is a hardcore necessity. Because AI algorithms will fill in and sift potential candidates matching your recruitment needs, Metaverse will reform your onboarding process by providing a virtually immersive and simulated workspace for the candidates. And even better with gamification! Do you know how? Well, you guessed it right! In short there is no other way around but implement, integrate and inch in the via the futuristic fusion of Metaverse and AI. digital transformation Conclusion So, if you’re still second guessing on integrating Metaverse with AI and taking on that digital facelift, you are running late. Because while this is a catalyst and not merely an upgrade. The integration changes everything and the entirety of the ecosystem surrounding us. It is certainly a star of its discipline and hitting the throttle for businesses across the genres. As Metaverse x AI is not a mere privilege, you can take a raincheck on but an iconic move that your business will be living on in the future. We at initially thought the next innovative leap is just chatbots and frameless screens. Only ‘few’ perceived the information, trusting its ability to lead the current digital transformation phase. Maybe, its not too late to be the ‘few’ and crave an innovative leap for your business too.

The Next Digital Leap: AI and Metaverse Integration for Businesses

Hey everyone! and just like you, I’ve been fascinated with the recent progress of artificial intelligence. Realizing that I needed to stay abreast with all the developments happening, I decided to embark on a personal journey of learning, thus was born! With this series, I will be learning about LLMs and share ideas, experiments, opinions, trends & learnings through my blog posts. You can follow along the journey on HackerNoon or my personal website . , I’m Nataraj 100 days of AI here here In one of the previous posts, we talked about finetuning and why it is important. In this post we will take a look at a specific kind of finetuning called Instruction Finetuning. Limitations of Pre-Trained Base Models: Pretrained base models like gpt-3 are trained on a vast amounts of data. In case of gpt-3 its all the data on the internet. Well we don’t know that for sure but most of these models are trained on internet scale data after considerable manual clean up and formatting. As they are trained the based models learn how to predict the next token and get really good at token prediction. But pure token prediction is not as useful as you would think. If you ask a pre-trained base model “ ” it will not reply with an answer but might complete the input sentence with “ “. So even though a model like gpt-3 is powerful at token prediction it will not work as a chatbot or a copilot. So how do we convert a pre-trained model to a useful chatbot like chat-gpt? The answer is finetuning, mainly a specific type of finetuning called “ “. What is the capital of Mexico? What is the capital of Columbia Instruction Finetuning What is instruction finetuning? Instruction finetuning also referred as “instruction-following” is a process to teach a pre-trained base model to behave like a chat bot. needs data sets which are in the form of question and answers. You can use public data sets or your companies data set which is in the form of Q&A. If your data set is not in the form of Q&A you can convert the data into Q&A using different techniques like Alpaca or using custom prompts on other LLMs. Note that instruction finetuning gives the model a new behavior of answering questions not just on the data that you use in finetuning, but this new behavior is applicable to the existing knowledge the model already has which makes finetuning a powerful technique. Instruction finetuning Instruction Finetuning Using Lamini: Lamini is an AI company that allows developers to deal with language models in an easy way abstracting away the complexity of hosting, training and other complicated aspects. Check out its full capabilities . We will use Lamini to train small launguage model called , which is an opensource model created by and do on it using a company dataset called Alpaca. here pythia Eleuther AI instruction finetuning Step 1: Initialize and load Instruction Finetuning dataset In this step lets initialize the required module and also look at the alpaca training data set. Here’s the code. import itertools
import jsonlines

from datasets import load_dataset
from pprint import pprint

from llama import BasicModelRunner
from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

## we are using alpaca data set, which is an open source fine tuning data set
instruction_tuned_dataset = load_dataset("tatsu-lab/alpaca", split="train", streaming=True)
m = 5
print("Instruction-tuned dataset:")
top_m = list(itertools.islice(instruction_tuned_dataset, m))
for j in top_m:
  print(j) This is how the instruction tuning data set looks like. It contains data in the form of questions and answers. Step 2: Hydrate the prompts In this step we take the data from the alpaca set and put them in to the prompts show below. prompt_template_with_input = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:"""

prompt_template_without_input = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:"""

## hydrate prompts - meaning add data to the above prompts
processed_data = []
for j in top_m:
  if not j["input"]:
    processed_prompt = prompt_template_without_input.format(instruction=j["instruction"])
  else:
    processed_prompt = prompt_template_with_input.format(instruction=j["instruction"], input=j["input"])

  processed_data.append({"input": processed_prompt, "output": j["output"]}) After doing this the data set will look as follows. We are basically taking the raw Q&A data and converting in to a format that makes sense to the LLM that when asked a question how should the response for that question should look like. We do this iteratively and store in a file. jsonl with jsonlines.open(f'alpaca_processed.jsonl', 'w') as writer:
    writer.write_all(processed_data) Step 3 – Non-Finetuned Output In step 1 & 2 we loaded raw data and hydrated it and stored in format. But Lamini has this hydrated data ready to go, so technically step 1 & 2 are not necessary. But it was needed to show to understand how instruction finetuning works. Let’s first see how a non-finetuned version of Pythia model would respond to a simple question. jsonl tokenizer = AutoTokenizer.from_pretrained("EleutherAI/pythia-70m") #70M parameter model that is not instruction tuned.
model = AutoModelForCausalLM.from_pretrained("EleutherAI/pythia-70m")

def inference(text, model, tokenizer, max_input_tokens=1000, max_output_tokens=100):
  # Tokenize
  input_ids = tokenizer.encode(
          text,
          return_tensors="pt",
          truncation=True,
          max_length=max_input_tokens
  )

  # Generate
  device = model.device
  generated_tokens_with_prompt = model.generate(
    input_ids=input_ids.to(device),
    max_length=max_output_tokens
  )

  # Decode
  generated_text_with_prompt = tokenizer.batch_decode(generated_tokens_with_prompt, skip_special_tokens=True)

  # Strip the prompt
  generated_text_answer = generated_text_with_prompt[0][len(text):]

  return generated_text_answer

## the 70M model doesnt have any company specific data, we will use the alpace data set from hosted on lamini and fine tune this model
# load alpaca dataset
finetuning_dataset_path = "lamini/lamini_docs"
finetuning_dataset = load_dataset(finetuning_dataset_path)
#print(finetuning_dataset)

test_sample = finetuning_dataset["test"][0]
print(test_sample)
print("untrained output sample")
print(inference(test_sample["question"], model, tokenizer)) This is the output I got. You will notice that the output is not helpful and the model is trying to do token completion and is not giving an actual answer. Step 4 – Instruction Finetuned Output Once we use the q&a data seen in the previous step is used to instruction fine tune the same model will start to behave like a chat bot and will provide more accurate answers to your questions both on the fineunted data but also the data that the model already consists off. Its almost like when a child learns a language the first time, he or she will now be able to express the feelings they already have along with the new things that they learnt becuase of the language training. Just like the pretrained version of model, the instruction finetuned model is also hosted on Lamini and can be infered with a command as show below. (Yes Lamini is great!) ## finetuned output
instruction_model = AutoModelForCausalLM.from_pretrained("lamini/lamini_docs_finetuned")
print("instruction finetuned output")
print(inference(test_sample["question"], instruction_model, tokenizer)) Here is what the output will look like. You will note that instead of the gibberish we have seen in the previous step we have a more accurate output. The goal with this post is to give an intro to and how it is used to make base models to more usable versions. In future posts I will dive deep into the actual process of doing instruction finetuning instruction finetuning. That’s it for Day 13 of 100 Days of AI. I write a newsletter called Above Average where I talk about the second order insights behind everything that is happening in big tech. If you are in tech and don’t want to be average, . subscribe to it Follow me on , for latest updates on 100 days of AI. If you are in tech you might be interested in joining my community of tech professionals . Twitter LinkedIn here Also appears . here

100 Days of AI, Day 13: How Instruction Finetuning Improves a Pre-trained LLM

এআই-এর 100 দিন, 13 দিন: কীভাবে নির্দেশনা ফাইনেটুনিং একটি প্রাক-প্রশিক্ষিত এলএলএম উন্নত করে

100 Tage KI, Tag 13: Wie die Feinabstimmung von Anweisungen ein vorab trainiertes LLM verbessert

100 días de IA, día 13: cómo el ajuste de la instrucción mejora un LLM previamente capacitado

100 jours d'IA, jour 13 : comment le réglage fin de l'instruction améliore un LLM pré-entraîné

एआई के 100 दिन, दिन 13: कैसे निर्देश फाइनट्यूनिंग एक पूर्व-प्रशिक्षित एलएलएम को बेहतर बनाता है

100 日間の AI、13 日目: 命令の微調整によって事前トレーニングされた LLM がどのように改善されるか

100일간의 AI, 13일차: 지침 미세 조정을 통해 사전 훈련된 LLM을 개선하는 방법

100 dias de IA, dia 13: como o ajuste fino de instruções melhora um LLM pré-treinado

100 дней искусственного интеллекта, день 13: как точная настройка инструкций улучшает предварительно обученного LLM

100 Günlük Yapay Zeka, 13. Gün: Talimatlarda İnce Ayarlama, Önceden Eğitimli Bir Yüksek Lisans Eğitimini Nasıl Geliştirir?

100 ngày của AI, Ngày 13: Cách tinh chỉnh hướng dẫn cải thiện LLM được đào tạo trước

人工智能 100 天，第 13 天：指令微调如何改进预训练的法学硕士

Built on Lance, an open-source columnar data format, LanceDB has some interesting features that make it attractive for AI/ML. For example, LanceDB supports explicit and implicit vectorization with the ability to handle various data types. LanceDB is integrated with leading ML frameworks such as and . Cooler still is LanceDB’s fast neighbor search which enables efficient retrieval of similar vectors using approximate nearest neighbor algorithms. All of these combine to create a vector database that is fast, easy to use and so lightweight it can be deployed anywhere. PyTorch TensorFlow LanceDB is capable of querying data in S3-compatible object storage. This combination is optimal for building high-performance, scalable, and cloud-native ML data storage and retrieval systems. MinIO brings performance and unparalleled flexibility across diverse hardware, locations, and cloud environments to the equation, making it the natural choice for such deployments. Upon completion of this tutorial, you will be prepared to use LanceDB and MinIO to joust with any data challenge. What is Lance? The file format is a columnar data format optimized for ML workflows and datasets. It is designed to be easy and fast to version, query, and use for training, and is suitable for various data types, including images, videos, 3D point clouds, audio, and tabular data. Additionally, it supports high-performance random access: with of 100 times faster than Parquet in queries. Lance’s speed is in part the result of being implemented in Rust, and its cloud-native design which includes features like zero-copy versioning and optimized vector operations. Lance Lance reporting benchmarks One of its is the ability to perform vector search, allowing users to find in under 1 millisecond and combine OLAP-queries with vector search. Other production applications for the lance format include edge-deployed low-latency vector databases for ML applications, large-scale storage, retrieval, and processing of multi-modal data in self-driving car companies, and billion-scale+ vector personalized search in e-commerce companies. Part of the appeal of the Lance file format is its compatibility with popular tools and platforms, such as Pandas, , Polars, and . Even if you don’t use LanceDB, you can still leverage the Lance file format in your data stack. key features nearest neighbors DuckDB Pyarrow Built for AI and Machine Learning Vector databases like LanceDB offer distinct advantages for AI and machine learning applications, thanks to their efficient and compute architectures and retrieval of high-dimensional vector representations of data. Here are some key use cases: decoupled storage Natural Language Processing (NLP): Find documents or passages similar to a query based on meaning, not just keywords. This powers chatbot responses, personalized content recommendations, and knowledge retrieval systems. Semantic Search: Understand and answer complex questions by finding relevant text passages based on semantic similarity. Question Answering: Discover latent topics in large text collections, useful for document clustering and trend analysis. Topic Modeling: Computer Vision: Search for similar images or videos based on visual content, crucial for content-based image retrieval, product search, and video analysis. Image and Video Retrieval: Improve the accuracy of object detection and classification models by efficiently retrieving similar training data. Object Detection and Classification: : Recommend similar videos based on the visual content of previously watched videos. Video Recommendation Among the plethora of vector databases on the market, LanceDB is particularly well suited for AI and machine learning, because it supports querying on S3- compatible storage. Your data is everywhere, your database should be everywhere too. Architecting for Success Using MinIO with LanceDB offers several benefits, including: : MinIO’s cloud-native design is built for scale and high-performance storage and retrieval. By leveraging MinIO's scalability and performance, LanceDB can efficiently handle large amounts of data, making it well-suited for modern ML workloads. Scalability and Performance : MinIO is highly available, immutable, and highly durable. This ensures that data stored in MinIO is protected against hardware failures and provides high availability and fault tolerance, which are crucial for data-intensive applications like LanceDB. High Availability and Fault Tolerance : Multi-site, active-active replication enables near-synchronous replication of data between multiple MinIO deployments. This robust process ensures high durability and redundancy, making it ideal for shielding data in mission-critical production environments. Active-active replication The combination of MinIO and LanceDB provides a high-performance scalable cloud-native solution for managing and analyzing large-scale ML datasets. Requirements To follow along with this tutorial, you will need to use . You can install the Docker Engine and Docker Compose binaries separately or together using Docker Desktop. The simplest option is to install Docker Desktop. Docker Compose Ensure that Docker Compose is installed by running the following command: docker compose version You will also need to install Python. You can download Python from . During installation, make sure to check the option to add Python to your system's PATH. here Optionally, you can choose to create a Virtual Environment. It's good practice to create a virtual environment to isolate dependencies. To do so, open a terminal and run: python -m venv venv To Activate the virtual environment: On Windows: .\venv\Scripts\activate On macOS/Linux: source venv/bin/activate Getting Started Begin by cloning the project from . Once done, navigate to the folder where you downloaded the files in a terminal window and run: here docker-compose up minio This will start up the MinIO container. You can navigate to ‘ ’ to take a look at the MinIO console. http://172.20.0.2:9001 Log in with the username and password . minioadmin:minioadmin Next, run the following command to create a MinIO bucket called . lance docker compose up mc This command performs a series of (mc) commands within a shell. MinIO Client Here's a breakdown of each command: ;: This command repeatedly attempts to configure a MinIO host named with the specified parameters (endpoint, access key, and secret key) until successful. During each attempt, it echoes a waiting message and pauses for 1 second. until (/usr/bin/mc config host add minio http://minio:9000 minioadmin minioadmin) do echo '...waiting...' && sleep 1; done minio This command forcefully removes (deletes) all contents within the bucket in MinIO. /usr/bin/mc rm -r --force minio/lance;: lance : This command creates a new bucket named in MinIO. /usr/bin/mc mb minio/lance; lance : This command sets the policy of the bucket to public, allowing public read access. /usr/bin/mc policy set public minio/lance; lance : This command ensures that the script exits with a status code of 0, indicating successful execution. exit 0; LanceDB Unfortunately, LanceDB does not have native S3 support, and as a result, you will have to use something like boto3 to connect to the MinIO container you made. As LanceDB matures we look forward to native S3 support that will make the user experience all the better. The sample script below will get you started. Install the required packages using pip. Create a file named requirements.txt with the following content: lancedb~=0.4.1
boto3~=1.34.9
botocore~=1.34.9 Then run the following command to install the packages: pip install -r requirements.txt You will need to change your credentials if your method of creating the MinIO container differs from the one outlined above. Save the below script to a file, e.g., . lancedb_script.py import lancedb
import os
import boto3
import botocore
import random

def generate_random_data(num_records):
    data = []
    for _ in range(num_records):
        record = {
            "vector": [random.uniform(0, 10), random.uniform(0, 10)],
            "item": f"item_{random.randint(1, 100)}",
            "price": round(random.uniform(5, 100), 2)
        }
        data.append(record)
    return data

def main():
    # Set credentials and region as environment variables
    os.environ["AWS_ACCESS_KEY_ID"] = "minioadmin"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "minioadmin"
    os.environ["AWS_ENDPOINT"] = "http://localhost:9000"
    os.environ["AWS_DEFAULT_REGION"] = "us-east-1"

    minio_bucket_name = "lance"

    # Create a boto3 session with path-style access
    session = boto3.Session()
    s3_client = session.client("s3", config=botocore.config.Config(s3={'addressing_style': 'path'}))

    # Connect to LanceDB using path-style URI and s3_client
    db_uri = f"s3://{minio_bucket_name}/"
    db = lancedb.connect(db_uri)

    # Create a table with more interesting data
    table = db.create_table("mytable", data=generate_random_data(100))

    # Open the table and perform a search
    result = table.search([5, 5]).limit(5).to_pandas()
    print(result)

if __name__ == "__main__":
    main() This script will create a Lance table from randomly generated data and add it to your MinIO bucket. Again, if you don’t use the method in the previous section to create a bucket you will need to do so before running the script. Remember to change the sample script above to match what you name your MinIO bucket. Finally, the script opens the table, without moving it out of MinIO, and uses Pandas to do a search and print the results. The result of the script should look similar to the one below. Remember that the data itself is randomly generated each time. vector      item  price  _distance
0  [5.1022754, 5.1069164]   item_95  50.94   0.021891
1   [4.209107, 5.2760105]  item_100  69.34   0.701694
2     [5.23562, 4.102992]   item_96  99.86   0.860140
3   [5.7922664, 5.867489]   item_47  56.25   1.380223
4    [4.458882, 3.934825]   item_93   9.90   1.427407 Expand on your Own There are many ways to build on this foundation offered in this tutorial to create performant, scalable and future-proofed ML/AI architectures. You have two cutting-edge and open-source building blocks in your arsenal – MinIO object storage and the LanceDB vector database –  consider this your winning ticket to the ML/AI . tournament Don’t stop here. LanceDB offers a wide range of and tutorials to expand on what you’ve built in this tutorial including a recently announced Udacity course on . Of particular interest is recipe to chat with your documents. We are all for breaking down barriers to getting the most from your data. recipes Building Generative AI Solutions with Vector Databases this Please show us what you’re building and should you need guidance on your noble quest don’t hesitate to email us at hello@minio.io or join our round table on . Slack

LanceDB: Your Trusted Steed in the Duel Against Data Complexity

ল্যান্সডিবি: ডেটা জটিলতার বিরুদ্ধে দ্বন্দ্বে আপনার বিশ্বস্ত স্টাইড

LanceDB: Ihr vertrauenswürdiges Pferd im Kampf gegen die Datenkomplexität

LanceDB: su corcel de confianza en el duelo contra la complejidad de los datos

LanceDB : votre monture de confiance dans le duel contre la complexité des données

लांसडीबी: डेटा जटिलता के विरुद्ध द्वंद्व में आपका विश्वसनीय घोड़ा

LanceDB: データの複雑さとの戦いにおいて信頼できる馬

LanceDB: 데이터 복잡성과의 결투에서 신뢰할 수 있는 군마

LanceDB: seu corcel confiável no duelo contra a complexidade dos dados

LanceDB: ваш надежный помощник в борьбе со сложностью данных

LanceDB: Veri Karmaşıklığına Karşı Düelloda Güvenilir Atınız

LanceDB: Chiến mã đáng tin cậy của bạn trong cuộc đấu tranh chống lại sự phức tạp của dữ liệu

LanceDB：对抗数据复杂性的值得信赖的骏马

The Great Barrier Reef in Australia is stunningly beautiful and the largest coral reef in the world, home to an incredible diversity of marine life. Unfortunately, the reef faces threats from crown-of-thorns starfish (COTS) that eat coral. To control COTS outbreaks, reef managers use a method called Manta Tow surveys where divers are towed behind a boat, visually assessing sections of the reef. However, this method has limitations in efficiency, data quality, and scalability. To improve COTS surveillance and control, the Great Barrier Reef Foundation started an innovation program. A key part is using underwater cameras to collect reef images and applying AI to detect COTS automatically. To develop the machine learning technology for this video-based surveying at scale, Australia's national science agency . Their goal is to create systems that can accurately and efficiently analyze large volumes of imagery to pinpoint COTS outbreaks across the vast Great Barrier Reef in near real-time. CSIRO has partnered with Google This would greatly assist conservation efforts and help secure the long-term protection of the reef ecosystem. In summary, AI-powered image analysis is being applied to automate detecting crown-of-thorns starfish from footage of the enormous Great Barrier Reef, enabling better monitoring and control at massively increased efficiency and scale to safeguard the reef. The is to accurately identify starfish in real-time by building an object detection model trained on underwater videos of coral reefs. In this blog post, we will build an ML pipeline to analyze the reef’s images and predict the presence and position of crown-of-thorns starfish. goal of this competition We will be using - an Opensource package that uses data-centric AI to clean up the issues in the image data and , , and - used for computer vision tasks like objection detection and image classification. Cleanvision Tensorflow KerasCV YOLOv8 !pip install -q cleanvision keras-cv kaggle As the data size is huge, we are directly using Kaggle to download and run the tutorial. For that, we need to configure your Kaggle creds and give you the appropriate permissions; ! read more here ! cp kaggle.json ~/.kaggle/
! chmod 600 ~/.kaggle/kaggle.json Downloading and Extracting the Kaggle Dataset !kaggle competitions download -c tensorflow-great-barrier-reef !unzip /content/tensorflow-great-barrier-reef.zip Using Cleanvision to Clean Up the Data CleanVision is an open-source data-centric AI package that automatically identifies problematic images that could negatively impact computer vision systems. It scans image datasets and detects common data issues like blur, inadequate lighting, duplicate images, etc. Fixing these problems in the training data is an essential first step before developing machine learning models to ensure their reliability. CleanVision provides a simple unified interface - just run the same few lines of Python code to audit any image collection, regardless of size or origin. This makes it extremely easy to clean dirty image data. By proactively flagging low-quality, duplicates, and other undesirable images upfront, CleanVision helps save significant time and improves outcomes when building computer vision applications. It's a fast way to boost data hygiene as a prerequisite before applying machine learning to image-based tasks. from cleanvision import Imagelab

# Specify path to folder containing the image files in your dataset
imagelab = Imagelab(data_path="train_images/")

# Automatically check for a predefined list of issues within your dataset
imagelab.find_issues()

# Produce a neat report of the issues found in your dataset
imagelab.report() import os
for st in imagelab.info['exact_duplicates']['sets']:
  os.remove(st[0]) Importing Relevant Libraries import pandas as pd
# show images inline
%matplotlib inline

import keras
import tensorflow

# import miscellaneous modules
import matplotlib.pyplot as plt
import cv2
import os
import numpy as np
import time

import tensorflow as tf Preparing Dataset for Preprocessing We will read the dataset and then preprocess it to be able to use it for model training. We begin by loading the train CSV file and converting the annotations to integer format from the original JSON format. df_train = pd.read_csv("train.csv")

df_train=df_train.loc[df_train["annotations"].astype(str) != "[]"]
df_train['annotations'] = df_train['annotations'].apply(eval)

df_train['image_path'] = "train_images/video_" + df_train['video_id'].astype(str) + "/" + df_train['video_frame'].astype(str) + ".jpg"
df_extrain=df_train.explode('annotations') # Single annotation per row
df_extrain.reset_index(inplace=True)
df_extrain.head() df_extrain_main=pd.DataFrame(pd.json_normalize(df_extrain['annotations']), columns=['x', 'y', 'width', 'height']).join(df_extrain)
df_extrain_main['class']=0
df_extrain_main=df_extrain_main[['image_path','x','y','width','height','class','video_id','video_frame']]
df_extrain_main.head(10) After the CSV processing, we try to split the CSV into a dataset and use the appropriate bounding box format, classes, and image paths to create the preliminary dataset ready to be processed further. def create_tf_example(rowss,data_df):


    xmins = []
    xmaxs = []
    ymins = []
    ymaxs = []

    # Convert ---> [xmin,ymin,width,height] to [xmins,xmaxs,ymins,ymaxs]
    xmin = rowss['x']
    xmax = rowss['x']+rowss['width']
    ymin = rowss['y']
    ymax = rowss['y']+rowss['height']


    #main_data.append((rowss['image_path'],xmins,xmaxs,ymins,ymaxs))
    return rowss['image_path'],xmin,ymin,xmax,ymax from PIL import Image, ImageDraw
paths = []
bboxes = []
classes = []
for index, row in df_extrain_main.iterrows():
            if index % 1000 == 0:
                print('Processed {0} images.'.format(index))
            image_path,xmins,ymins,xmaxs,ymaxs=create_tf_example(row,df_extrain_main)
            paths.append(image_path)
            bboxes.append([[float(xmins),float(ymins),float(xmaxs),float(ymaxs)]])
            classes.append([0]) Here, we are using to create ragged tensors from the and classes lists. A ragged tensor is a type of tensor that can handle varying lengths of data along one or more dimensions. tf.ragged.constant bbox This is useful when dealing with data that has variable-length sequences, such as text or time series data. In this case, the and classes lists have different lengths for each image, depending on the number of objects in the image and the corresponding bounding boxes and classes. To handle this variability, ragged tensors are used instead of regular tensors. bbox Later, these ragged tensors are used to create a using the method. This method creates a dataset from the input tensors by slicing them along the first dimension. tf.data.Dataset from_tensor_slices By using ragged tensors, the dataset can handle varying lengths of data for each image and provide a flexible input pipeline for further processing. bbox = tf.ragged.constant(bboxes)
classes = tf.ragged.constant(classes)
image_paths = tf.ragged.constant(paths)

data = tf.data.Dataset.from_tensor_slices((image_paths, classes, bbox)) num_val = int(bbox.shape[0] * 0.2)

# Split the dataset into train and validation sets
val_data = data.take(num_val)
train_data = data.skip(num_val) includes pre-trained models for popular computer vision datasets, such as , which can be used for transfer learning. KerasCV also provides a range of visualization tools for inspecting the intermediate representations learned by the model and for visualizing the results of object detection and segmentation tasks. KerasCV ImageNet, COCO, and Pascal VOC In this particular notebook, we are using YOLOV8, so we have to format the dataset which is in a compatible format according to the YOLOV8 model. The bounding box input format should be as follows. bounding_boxes = {
    # num_boxes may be a Ragged dimension
    'boxes': Tensor(shape=[batch, num_boxes, 4]),
    'classes': Tensor(shape=[batch, num_boxes])
} The dictionary has two keys, ' ' and ' ', each of which maps to a TensorFlow or Tensor object. The ' ' Tensor has a shape of , where batch is the number of images in the batch and is the maximum number of bounding boxes in any image. boxes classes RaggedTensor boxes [batch, num_boxes, 4] num_boxes The 4 represents the four values needed to define a bounding box: xmin, ymin, xmax, ymax. The 'classes' Tensor has a shape of , where each element represents the class label for the corresponding bounding box in the ' ' Tensor. The num_boxes dimension may be ragged, which means that the number of boxes may vary across images in the batch. [batch, num_boxes] boxes The final model input looks something like this: {"images": images, "bounding_boxes": bounding_boxes} def load_image(image_path):
    image = tf.io.read_file(image_path)
    image = tf.image.decode_jpeg(image, channels=3)
    return image


def load_dataset(image_path, classes, bbox):
    # Read Image
    image = load_image(image_path)
    bounding_boxes = {
        "classes": tf.cast(classes, dtype=tf.float32),
        "boxes": tf.cast(bbox, dtype=tf.float32),
    }
    return {"images": tf.cast(image, tf.float32), "bounding_boxes": bounding_boxes} Data Augmentation Applying effective data augmentation is critical yet challenging when building object detection models. Transformations like crops, flips, etc. must update the bounding box coordinates correctly. Doing this manually is complex and error-prone. KerasCV provides specialized layers to handle augmentation aware of bounding boxes. It offers a wide range of transformations that automatically adjust bboxes to match the augmented images. This integrated bounding box handling enables easily incorporating powerful augmentation into object detection pipelines with KerasCV. By leveraging tf.data pipelines, the augmentation can be done on-the-fly during training. With KerasCV's robust bbox-aware augmentation capabilities, developers can achieve more diverse and useful training data and improved generalization in object detection models, while avoiding cumbersome manual handling of coordinates. The layers handle that complexity behind the scenes. augmenter = keras.Sequential(
    layers=[
        keras_cv.layers.RandomFlip(mode="horizontal", bounding_box_format="xyxy"),
        keras_cv.layers.RandomShear(
            x_factor=0.2, y_factor=0.2, bounding_box_format="xyxy"
        ),
        keras_cv.layers.JitteredResize(
            target_size=(640, 640), scale_factor=(0.75, 1.3), bounding_box_format="xyxy"
        ),
    ]
)
resizing = keras_cv.layers.JitteredResize(
    target_size=(640, 640),
    scale_factor=(0.75, 1.3),
    bounding_box_format="xyxy",
) Training Dataset With Augmentation We are preparing the training dataset with augmentation for our object detection model. We start by setting the to 4, and then use the function to load the data. batch_size load_dataset Next, we shuffle the data to ensure that our model doesn't overfit on a particular kind of data. We then use the function to batch the data with a fixed size of 4 and drop any remaining data. ragged_batch Finally, we apply the function to the dataset to perform data augmentation, which helps to increase the diversity of the dataset and improve the model's accuracy. augmenter The parameter is set to to allow TensorFlow to dynamically tune the number of parallel calls to improve performance. num_parallel_calls tf.data.AUTOTUNE BATCH_SIZE =4
train_ds = train_data.map(load_dataset, num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.shuffle(BATCH_SIZE * 4)
train_ds = train_ds.ragged_batch(BATCH_SIZE, drop_remainder=True)
train_ds = train_ds.map(augmenter, num_parallel_calls=tf.data.AUTOTUNE) Validation Dataset In this code block, we are creating a validation dataset by mapping the 'load_dataset' function to the 'val_data' set. This function helps in loading the data from the dataset and preprocessing it for the model. We are also setting 'num_parallel_calls' to 'tf.data.AUTOTUNE' to enable parallelism for faster processing. Next, we are shuffling the validation dataset by a factor of 'BATCH_SIZE * 4' to increase randomness and prevent overfitting. Finally, we are creating a 'ragged_batch' of 'BATCH_SIZE' with 'drop_remainder=True' to ensure that all batches have the same size. This helps in making the model more efficient and consistent during training. val_ds = val_data.map(load_dataset, num_parallel_calls=tf.data.AUTOTUNE)
val_ds = val_ds.shuffle(BATCH_SIZE * 4)
val_ds = val_ds.ragged_batch(BATCH_SIZE, drop_remainder=True) Bounding Box Visualisation We use the library for bounding box visualization. It imports the necessary packages and defines a class mapping for the dataset. The function takes in the dataset inputs and uses the function to display the images with bounding boxes overlaid. You can see that via and datasets. keras_cv visualize_dataset() plot_bounding_box_gallery() train_ds val_ds import keras_cv
from keras_cv import bounding_box
from keras_cv import visualization
class_mapping = {0:'fish'}

def visualize_dataset(inputs, value_range, rows, cols, bounding_box_format):
    inputs = next(iter(inputs.take(1)))
    images, bounding_boxes = inputs["images"], inputs["bounding_boxes"]
    visualization.plot_bounding_box_gallery(
        images,
        value_range=value_range,
        rows=rows,
        cols=cols,
        y_true=bounding_boxes,
        scale=5,
        font_scale=0.7,
        bounding_box_format=bounding_box_format,
        class_mapping=class_mapping,
    )


visualize_dataset(
    train_ds, bounding_box_format="xyxy", value_range=(0, 255), rows=2, cols=2
)

visualize_dataset(
    val_ds, bounding_box_format="xyxy", value_range=(0, 255), rows=2, cols=2
) Next, we unpack the dataset to be able to feed it into the model training function of the Keras API. def dict_to_tuple(inputs):
    return inputs["images"], inputs["bounding_boxes"]


train_ds = train_ds.map(dict_to_tuple, num_parallel_calls=tf.data.AUTOTUNE)
train_ds = train_ds.prefetch(tf.data.AUTOTUNE)

val_ds = val_ds.map(dict_to_tuple, num_parallel_calls=tf.data.AUTOTUNE)
val_ds = val_ds.prefetch(tf.data.AUTOTUNE) Model Building is the newest iteration of the popular YOLO (You Only Look Once) family of models used for computer vision tasks like object detection and image classification. It was created by Ultralytics, building on their previous YOLOv5 model. YOLOv8 Compared to prior versions, YOLOv8 integrates numerous architectural upgrades for improved accuracy and performance. The creators also aimed to enhance the overall developer experience. As the latest release in the industry-leading YOLO lineup, YOLOv8 represents the current state-of-the-art in domains like object detection. It incorporates both algorithmic advances to push the boundaries of CV capability and usability improvements for practitioners. Keras-CV offers multiple models- , , , , and . For this example, we choose the backbone and load it with the pre-trained weights. YOLOV8 YOLOv8n YOLOv8s YOLOv8m YOLOv8l YOLOv8x YOLOV8s coco Next, we build a model using the which accepts a feature extractor as the backbone argument, a num_classes argument that specifies the number of object classes to detect based on the size of the class_mapping list, a bounding_box_format argument that informs the model of the format of the bbox in the dataset, and finally, the feature pyramid network (FPN) depth is specified by the argument. YOLOV8 YOLOV8Detector fpn_depth backbone = keras_cv.models.YOLOV8Backbone.from_preset(
    "yolo_v8_s_backbone_coco"  # We will use yolov8 small backbone with coco weights
)
yolo = keras_cv.models.YOLOV8Detector(
    num_classes=1,
    bounding_box_format="xyxy",
    backbone=backbone,
    fpn_depth=1,
) Compiling the Model The training process of a YOLOv8 model involves the use of two types of losses - classification loss and box loss. The classification loss measures the difference between the predicted class probabilities and the true class probabilities for each detected object. Binary cross entropy loss is utilized in this case, as each detected item either belongs to a particular class or does not. This loss function helps optimize the model for accurate object classification. On the other hand, the box loss calculates the dissimilarity between the predicted bounding boxes and the ground truth boxes. YOLOv8 uses Complete IoU (CIoU) instead of basic IoU for box loss calculation. CIoU considers additional factors like the boxes' aspect ratio, center distance, and scale. This helps better represent box similarity by looking at more box attributes beyond just overlap. By jointly optimizing for object classification and precise localization during training, the classification and box losses help YOLOv8 generate highly accurate object detection by minimizing discrepancies between outputs and label data across classes and bounding box coordinates. optimizer = tf.keras.optimizers.Adam(
    learning_rate=10e-3,
)

yolo.compile(
    optimizer=optimizer, classification_loss="binary_crossentropy", box_loss="ciou"
) Training the Model During each epoch, the model iteratively adjusts its parameters to minimize the difference between its predicted output and the actual output. This process helps the model learn how to accurately detect crown-of-thorns starfish in the underwater videos of coral reefs. The number of epochs can be adjusted based on the training dataset size, complexity of the model, and desired accuracy. After training, the model can be used to predict the presence and position of starfish in real-time, helping researchers and conservationists monitor and control COTS outbreaks more efficiently and effectively. yolo.fit(
    train_ds,
    epochs=3,
) Conclusion The integration of , , , and in this project exemplifies the power and versatility of AI in environmental conservation, particularly for the Great Barrier Reef. Cleanvision TensorFlow KerasCV YOLOv8 By leveraging the strengths of these advanced tools, we've developed a robust model capable of accurately identifying crown-of-thorns starfish in complex underwater environments. This not only marks a significant step forward in protecting one of the world's natural wonders but also demonstrates the immense potential of AI in addressing ecological challenges.

How to Use TensorFlow and Cleanvision to Detect Starfish Threats in the Great Barrier Reef

311 Stories To Learn About Noonies

Redux has gained immense popularity for managing application state in a predictable and maintainable manner. This comprehensive guide walks through the process of implementing Redux in an iOS app, empowering developers to build scalable and efficient applications. From understanding core principles to practical implementation, this guide serves as a robust resource for adopting Redux in iOS development : the core principles - single source of truth, state immutability, and unidirectional data flow, where: Understanding Redux Architecture Store: Define the single source of truth for the application state Actions: Describe events that trigger state changes by reducer and trigger middleware Reducers: Pure functions that handle actions and update the state Middlewares: Interaction with services and side effects and the result of a new action . Create a new iOS project. Initialize the project structure, including folders Core, Flow, MW, Store, Services, where: Setting Up the Project Core for the basic architecture components and all additional general utilities Flow for UI components Views and their Service something like Controller/Presenter/View Model MW for Middlewares, DTO’s Store for Actions, Reducers, State Service for all components that are connected to the outside world something like File Manager/DB/Network/… . At first make protocol for actions and at the same time include reducer. Define actions that represent events triggering state changes. Organize actions into separate files based on functionality. Defining Actions Implementing Reducers. Create reducers that handle actions and update the state immutably. Combine reducers to manage different parts of the state protocol Action {
    associatedtype StateType: Equatable
    func reduce(_ state: inout StateType)
} enum MainAction: Action {
    case user(UserAction)
    
    func reduce(_ state: inout MainState) {
        switch self {
        case let .user(action): action.reduce(&state)
        }
    }
} enum UserAction: Action {
    case getUserData
    case receiveUser(UserModel)
    
    func reduce(_ state: inout MainState) {
        switch self {
        case .getUserData: state.user = .loading
        case let .receiveUser(model): state.user = .result(model)
        }
    }
} . Make base elements and protocol for Middleware. Implement middleware to manage asynchronous actions. And create DTO with mapping to State model Managing Asynchronous Operations struct Box<Act> where Act: Action {
    let run: (@escaping (Act) -> Void) -> Void
    
    init(_ run: @escaping (@escaping (Act) -> Void) -> Void) {
        self.run = run
    }
}

extension Box {
    static func throwTask(
        _ run: @escaping (@escaping (Act) -> Void) async throws -> Void,
        catchError: @escaping (@escaping (Act) -> Void, Error) -> Void = { _, _ in }
    ) -> Self {
        Self { dispatcher in
            Task {
                do {
                    try await run(dispatcher)
                } catch {
                    catchError(dispatcher, error)
                }
            }
        }
    }
}

extension Box {
    static var zero: Self { Self { _ in }}
    
    static func +=(lhs: inout Self, rhs: Self) {
        let tmp = lhs
        lhs = Self { dispatcher in
            tmp.run(dispatcher)
            rhs.run(dispatcher)
        }
    }
} protocol Middleware {
    associatedtype Act where Act: Action
    func run(state: Act.StateType, action: Act) -> Box<Act>
}

extension Middleware {
    var asAnyMiddleware: AnyMiddleware<Act> {
        AnyMiddleware(self.run(state:action:))
    }
} struct AnyMiddleware<Act>: Middleware where Act: Action {
    private let runer: (Act.StateType, Act) -> Box<Act>
    
    init(_ run: @escaping (Act.StateType, Act) -> Box<Act>) {
        self.runer = run
    }
    
    func run(state: Act.StateType, action: Act) -> Box<Act> {
        runer(state, action)
    }
} struct UserMW: Middleware {
    private let net: UserNetProvider
    
    init(net: UserNetProvider) {
        self.net = net
    }
    
    func run(state: MainState, action: MainAction) -> Box<MainAction> {
        switch action {
        case let .user(userAction): return handleUser(state: state.user, action: userAction)
        default: return .zero
        }
    }
    
    func handleUser(state: ModelWrapper<UserModel>, action: UserAction) -> Box<MainAction> {
        switch action {
        case .getUserData: return Box.throwTask { dispatcher in
            let userDTO: UserDTO = try await net.getUserData()
            dispatcher(.user(.receiveUser(userDTO.convert)))
        } catchError: { dispatcher, error in
            // Handle error
        }
        default: return .zero
        }
    }
} struct UserDTO: Decodable {
    let name: String
    let age: Int
    
    var convert: UserModel {
        UserModel(name: name, age: age)
    }
} . Create State and Model wrapper for implementing all states. Create a store to manage the application state. Define the initial state and configure the store. Implementing the Store Of course Store implementation is very simple without corner cases enum ModelWrapper<Model> where Model: Equatable {
    case initialize
    case loading
    case error(Error)
    case result(Model)
}

extension ModelWrapper: Equatable {
    private var equatabeValue: String {
        return switch self {
        case .initialize: "initialize \(String(describing: Model.self))"
        case .loading: "loading \(String(describing: Model.self))"
        case let .error(error): error.localizedDescription
        case let .result(model): String(describing: model)
        }
    }
    
    static func == (lhs: ModelWrapper<Model>, rhs: ModelWrapper<Model>) -> Bool {
        lhs.equatabeValue == rhs.equatabeValue
    }
} struct MainState: Equatable {
    public internal(set) var user: ModelWrapper<UserModel> = .initialize
} struct UserModel: Equatable {
    let name: String
    let age: Int
} actor Store<Act> where Act: Action {
    nonisolated var statePublisher: AnyPublisher<Act.StateType, Never> {
        stateSubject.eraseToAnyPublisher()
    }
    
    private var state: Act.StateType {
        didSet {
            stateSubject.send(state)
        }
    }
    private let stateSubject: CurrentValueSubject<Act.StateType, Never>
    
    private let middlewares: [AnyMiddleware<Act>]
    
    init(initState: Act.StateType, middlewares: [AnyMiddleware<Act>]) {
        self.state = initState
        self.stateSubject = CurrentValueSubject(initState)
        self.middlewares = middlewares
    }
    
    func dispatch(action: Act) {
        var box = Box<Act>.zero
        
        for mw in middlewares {
            box += mw.run(state: state, action: action)
        }
        
        action.reduce(&state)
        
        box.run(dispatch(action:))
    }
} . Integrate Redux with view components to read state from the store. Create simple View with ViewController and dispatch actions to trigger state updates. Connecting Components to Redux class ReduxVC<Act>: UIViewController where Act: Action {
    private let store: Store<Act>
    private var box = Set<AnyCancellable>()
    
    init(store: Store<Act>) {
        self.store = store
        super.init(nibName: nil, bundle: nil)
    }
    
    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }
    
    func sink<S: Equatable>(map: KeyPath<Act.StateType, S>, _ sink: @escaping (_ storeState: S) -> Void) {
        store
            .statePublisher
            .map(map)
            .removeDuplicates()
            .receive(on: DispatchQueue.main)
            .sink(receiveValue: sink)
            .store(in: &box)
    }
    
    func dispatch(_ action: Act) {
        Task { await store.dispatch(action: action) }
    }
} final class UserScreenView: UIView {
    
    private let spiner = {
        $0.isHidden = true
        $0.color = .black
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UIActivityIndicatorView(style: .medium))
    
    private let dataView = {
        $0.isHidden = true
        $0.backgroundColor = .lightGray
        $0.clipsToBounds = true
        $0.layer.cornerRadius = 10
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UIView())
    
    private let errorView = {
        $0.isHidden = true
        $0.backgroundColor = .lightGray
        $0.clipsToBounds = true
        $0.layer.cornerRadius = 10
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UIView())
    
    private let nameLabel = {
        $0.font = .systemFont(ofSize: 22, weight: .regular)
        $0.textColor = .black
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UILabel())
    
    private let ageLabel = {
        $0.font = .systemFont(ofSize: 22, weight: .regular)
        $0.textColor = .black
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UILabel())
    
    private let errorLabel = {
        $0.font = .systemFont(ofSize: 22, weight: .regular)
        $0.textColor = .systemRed
        $0.translatesAutoresizingMaskIntoConstraints = false
        return $0
    }(UILabel())
    
    init() {
        super.init(frame: .zero)
        backgroundColor = .white
        
        addSubview(spiner)
        addSubview(dataView)
        dataView.addSubview(nameLabel)
        dataView.addSubview(ageLabel)
        addSubview(errorView)
        errorView.addSubview(errorLabel)
        
        NSLayoutConstraint.activate([
            spiner.centerXAnchor.constraint(equalTo: centerXAnchor),
            spiner.centerYAnchor.constraint(equalTo: centerYAnchor),
            spiner.heightAnchor.constraint(equalToConstant: 30),
            spiner.widthAnchor.constraint(equalToConstant: 30),
            
            dataView.centerXAnchor.constraint(equalTo: centerXAnchor),
            dataView.centerYAnchor.constraint(equalTo: centerYAnchor),
            nameLabel.leadingAnchor.constraint(equalTo: dataView.leadingAnchor, constant: 22),
            nameLabel.trailingAnchor.constraint(equalTo: dataView.trailingAnchor, constant: -22),
            nameLabel.topAnchor.constraint(equalTo: dataView.topAnchor, constant: 22),
            ageLabel.widthAnchor.constraint(equalTo: nameLabel.widthAnchor),
            ageLabel.centerXAnchor.constraint(equalTo: dataView.centerXAnchor),
            ageLabel.topAnchor.constraint(equalTo: nameLabel.bottomAnchor, constant: 22),
            ageLabel.bottomAnchor.constraint(equalTo: dataView.bottomAnchor, constant: -22),
            
            errorView.centerXAnchor.constraint(equalTo: centerXAnchor),
            errorView.centerYAnchor.constraint(equalTo: centerYAnchor),
            errorLabel.leadingAnchor.constraint(equalTo: errorView.leadingAnchor, constant: 22),
            errorLabel.trailingAnchor.constraint(equalTo: errorView.trailingAnchor, constant: -22),
            errorLabel.topAnchor.constraint(equalTo: errorView.topAnchor, constant: 22),
            errorLabel.bottomAnchor.constraint(equalTo: errorView.bottomAnchor, constant: -22)
        ])
    }
    
    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }
    
    func loadingData() {
        dataView.isHidden = true
        errorView.isHidden = true
        spiner.isHidden = false
        spiner.startAnimating()
    }
    
    func errorSetup(_ error: String) {
        dataView.isHidden = true
        spiner.isHidden = true
        errorView.isHidden = false
        spiner.stopAnimating()
        
        errorLabel.text = error
    }
    
    func dataSetup(_ model: UserModel) {
        dataView.isHidden = false
        spiner.isHidden = true
        errorView.isHidden = true
        spiner.stopAnimating()
        
        nameLabel.text = model.name
        ageLabel.text = String(model.age)
    }
} final class UserScreenVC: ReduxVC<MainAction> {

    let mainView = UserScreenView()
    
    override func loadView() {
        self.view = mainView
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        sink(map: \.user) { [unowned self] state in
            switch state {
            case .initialize: break
            case .loading: mainView.loadingData()
            case let .error(err): mainView.errorSetup(err.localizedDescription)
            case let .result(model): mainView.dataSetup(model)
            }
        }
        
        dispatch(.user(.getUserData))
    }
} And finally assembly all components together into SceneDelegate guard let windowScene = (scene as? UIWindowScene) else { return }

let netProvider = NetProvider()

let store = Store<MainAction>(
    initState: MainState(),
    middlewares: [
        UserMW(net: UserNetProvider(netProvider: netProvider)).asAnyMiddleware
    ]
)

window = UIWindow(frame: UIScreen.main.bounds)
window?.windowScene = windowScene
window?.rootViewController = UserScreenVC(store: store)
window?.makeKeyAndVisible() All structural folders Core, Flow, MW, Store, Services you can separate into separate elements e.g. Frameworks/Swift Packages/Libraries or something like this Conclusion: In conclusion, the adoption of Redux architecture in iOS app development offers a structured, scalable, and efficient approach to managing application state. This comprehensive guide has illuminated the core principles of Redux, emphasizing the importance of a single source of truth, state immutability, and unidirectional data flow. The step-by-step exploration of implementing Redux architecture has covered various key components

Building Testable and Maintainable iOS Apps with Redux

361 Stories To Learn About Big Data

I am building an open-source, highly scalable, and low-cost . URL shortener with Cloudflare KV By low-cost I mean, just $5 per month on Cloudflare worker and a free plan for basic requirement Idea At DaySchedule, we had many customers upvote a URL shortener feature with auto-expirable links to book an appointment instead of sharing their original URL. So, I decided to try Cloudflare for this project - By leveraging Cloudflare KV's capabilities, the goal was to develop a solution that offered not only URL shortening but also customization options, analytics, and scalability, all while maintaining a significantly lower cost compared to our established services on AWS. Features Expirable links using option on KV cacheTtl Custom domain using Cloudflare for SaaS Analytics using Worker analytics API I used by for API routing, and it's blazing fast and offers Cloudflare pages template to build Edge projects quickly. hono.dev @yusukebe Here is an example of from their docs - server.ts import { Hono } from 'hono'
const app = new Hono()

app.get('/', (c) => c.text('Hono!'))

export default app Controller The API is all I need to create, update, and manage the short links. /links const links = new Hono<{ Bindings: Bindings }>();

links.post(
  '/',
  validator('json', (value, c) => {
    const parsed = linkSchema.safeParse(value);
    if (!parsed.success) {
      return c.json(parsed, 400);
    }
    return parsed.data as ShortLink;
  }),
  async (ctx) => {
    const data = ctx.req.valid('json');
    // Set expire_at if given, or fallback to 1 month expiry
    const expireAt = data.expire_at
      ? dayjs(data.expire_at).unix()
      : dayjs().add(1, 'month').unix();
    if (expireAt <= dayjs().unix()) {
      return ctx.json(
        { message: 'The expire_at must be greater then current date time' },
        400
      );
    }

    let key = data.key || nanoid(7);
    let exists = await ctx.env.SHORTLINKS.get(key);
    while (exists) {
      key = nanoid(7);
      exists = await ctx.env.SHORTLINKS.get(key);
    }
    await ctx.env.SHORTLINKS.put(key, JSON.stringify(data), {
      expiration: expireAt,
    });
    return ctx.json(
      { ...data, key: key, short_url: `https://idm.in/${key}` },
      200
    );
  }
); Code explanations - This code is request to create short links. Here's a breakdown of the code to explain what I am doing: POST It creates an instance of the Hono object. const links = new Hono<{ Bindings: Bindings }>(): Validation with zod - The function is used as a middleware for validating the incoming JSON payload defined in . validator('json', (value, c) => {...}): linkSchema Set expiration: Checks if an expiry date is provided in the payload. If not, it sets an expiry date one month from the current date and time. Generates a unique key for the link using or uses the provided key from the payload if available. nanoid(7) Stores the link data (converted to a string) in the Cloudflare KV ( ) with an expiration time based on the calculated expiry date. JSON SHORTLINKS Demo The demo is available on , it's free to use and open-sourced on Github to . IDM build your custom URL shortener You can clone the repo to deploy on your Cloudflare account. Don’t forget to on GitHub to show your support :-) Star the repository Also published here.

How I Built a Low-Cost URL Shortener With Cloudflare KV

Dashboard software has become a necessary tool for businesses. With all the information you must gather and monitor about your business, you may wonder how to get the most out of it. Actionable insight can help you gain a competitive edge, considering the gap between data and business value. In fact, in 2023, achieved measurable value from and analytics investments, which reinforces the need for data-driven decision-making. 91.9% of organizations data to ensure you understand your data and benefit from it. Dashboard software merges data from linked databases and provides a customizable and interactive platform for data visualization Additionally, it empowers everyone in your company, from a data analyst to a junior manager, allowing them to visualize, present, and use data in any way that makes sense to them. In this article, we’ll explore the most eminent benefits of dashboard software and how you can leverage them to get the most out of your investment. 1. Effective Data Visualization Businesses gather huge amounts of data. Understanding such amounts of data can be challenging and time-consuming, to say the least, necessitating the need for tools that can help organize your data in a manner you can comprehend. When this is not possible, your business may experience challenges in making decisions and executing them in time. How can you make decisions based on data communicated ineffectively? Similarly, . This leads to a lot of time wasted trying to organize and present huge amounts of data. without data visualization, your employees may not know where to focus their attention to improve productivity One of the benefits of dashboard software is its ability to transform your business data into easy-to-understand graphs, images, and charts, which are essential considering that the human brain processes visual information more readily than textual information. Actually, the use of visual language is proven to enhance effectiveness and efficiency, leading to 24% shorter meetings. It is safe to conclude that this statistic is only possible because visualization enables data to be easily communicated and understood. The software interprets and simplifies the data for you to help generate true value for your business. It becomes easier to make decisions, recognize patterns and errors, and improve data analysis. GenerationE is an IT solutions provider that captures and automates best practices for IT operations by up to 85%. InetSoft provided a visualization layer for GenerationE, helping the company and saving its team the effort of charting and analyzing log data in external reporting systems. monitor the performance of automated processes 2. Improves Decision Making Great data, when analyzed correctly, can help enhance decision-making. It is important to gather and analyze business data to understand key business metrics and your performance to help you make better decisions to upscale. Without good dashboard software, your business may experience challenges collecting and managing data as it is scattered across multiple databases. Even if you can access this data from siloed systems, moving and analyzing it manually takes time and can impede accurate reporting and fast decision-making. Slow and inefficient decision-making can negatively impact your business in 2024 by costing you missed opportunities, time, and money. Dashboard software merges data from different sources and provides immediate access to at-a-glance information, ensuring you make decisions quickly to avoid missing out on opportunities. Additionally, with clear, actionable insights, you can make more accurate data-driven decisions and create business-boosting strategies to give you a competitive edge. In fact, Insight-driven businesses are growing at an average of , according to a report by Forrester. 30% each year This figure summarizes the when it comes to decision-making. Data in the digital era is extremely critical, and it helps to have the right tools to help you leverage. benefits of dashboard software Kreditech is a financial company based in Germany that offers clients loans based on their creditworthiness. Data analysis and decision-making are therefore critical. After incorporating Datapine, Kreditech was able to decentralize decision-making and achieve faster and smarter data-driven decisions. Thanks to the software’s friendly interface, users were able to work with zero IT overhead. 3. Increases Business Efficiency Enhancing business efficiency can help grow your business. It is undeniable how critical business efficiency is to grow and survive in the current economic climate. Without dashboard software, your business risks inefficiencies due to siloed information, manual data entry processes, ineffective data organization, and presentation. To err is human, so inefficiencies can frequently occur without the right tools to streamline business intelligence operations. Manual processes can also lead to data errors that are more prevalent when dealing with large amounts of data. In fact, Gartner found out that organizations believe poor data quality to be responsible for per year in losses. an average of $15 million For starters, reducing manual tasks is a key component for successful business management and plays a huge role in increasing efficiencies. It is a no-brainer that automation technologies like dashboard software can achieve more than a human and deliver more efficient results through workflow automation. A dashboard software offers smart, autonomous data management solutions to help you save time and money by gathering quality insights. Additionally, dashboard software brings data from different sources into a central respiratory system, eliminating the need for multiple external solutions. Therefore, you can be able to take quick action on your data-driven insights to upscale your business. Parcero is a Columbian marketing agency that was looking for a solution to help save time on manual processes. After implementing Klipfolio, Parcero was able to automate client reporting and enhance efficiency. They recorded in repetitive advertising tasks and 73% in repetitive reporting tasks. 33% time savings 4. Empowers End-Users Dashboard software empowers your employees to be self-reliant. The amount of time wasted waiting for data analysts and IT teams to present and explain data is a lot. This is not to mention that it distracts them from focusing on their job to advance the business’s goals. Unfortunately, there remains a huge gap between having data and being able to use it. Even with huge amounts of data, most businesses need help to take action on the valuable insight because they cannot interpret it. In fact, struggle with utilizing their , according to Talend. 78% of executives data for decision-making Imagine how much you could achieve if your employees could access and interpret data on their own without relying on data analysts. Well, one of the in business intelligence is empowering end-users by providing a simplified and clear platform where they can access data and manipulate it in a way that makes sense to them. benefits of dashboard software This ensures enhanced productivity, prompt decision-making, and minimized costs thanks to the ability to get answers independently without needing help from the BI or IT team. This also allows your IT team to focus on their job without being distracted by frequent help requests. GMS is a leading distributor of construction products based in Georgia. The company was looking for an easy-to-implement solution to help users access valuable insights. Thanks to InetSoft, GMS users can based on data consolidated into interactive reports that are easy to read. make quick business decisions 5. Reduces Operating Costs Dashboard software can minimize business costs. Without a great dashboard software in 2024, data remains fragmented across multiple systems and databases, impeding prompt decision-making. Siloed data can result in expensive operational costs caused by inefficiencies, analysis, and manual data collection. . Without , your employees may spend hours fetching and trying to interpret data. Time is money, and when you are losing both, you could be laying a foundation for your business’s failure the right tools In fact, Airtable’s research revealed that knowledge workers reported spending nearly 29% of their week (11.6 hrs) searching for the key information they need to do their work. Conversely, implementing dashboard software provides a central repository that combines and organizes data from different sources to provide clear visibility of key metrics. . It also provides a unified view that lets you easily identify cost-saving optimization opportunities and budget allocation opportunities In addition to that, dashboard software capabilities such as automation help to eliminate manual and repetitive tasks that can be time-consuming. Consequently, this reduces the chances of inefficiencies and errors that could cost you money. Dashboard software significantly reduces operational expenses by identifying inefficiencies promptly, optimizing resource allocation, and making data-driven decisions, ultimately contributing to substantial cost savings. Just Spices is a German-based company that manufactures spices. The company had been investing a lot of time in extracting and analyzing data for a while. With Klipfolio, Just Spices as they could access and visualize key metrics that show different departments where help is needed instead of having people extract data. saved time and money Conclusion In the digital age, data is extremely essential for businesses. How you use the business data you gather can influence the success of your business. However, with vast amounts of data, it can be challenging even for experts to organize and leverage data to get the best out of it. Dashboard software benefits such as reduced costs, enhanced efficiency, and clear visibility into key metrics can set your business apart. Actually, according to Capgemini, say its company’s Big Data initiatives are profitable. 27% of business executives Investing in a great dashboard software in 2024 can empower small businesses with enterprise-level insight without exaggerated costs.

5 Benefits of Dashboard Software for Business Intelligence in 2024

Build Your Own Chatbot For An Enhanced DevOps Experience

Stories, News & Mentions about DATAXPIE

Dataxpie’s Stories on HackerNoon

Dataxpie's News & Mentions