아이 스스로 즐기는 코코지! 아이들 발달에 도움을 주고 상상력을 키워주는 오디오 콘텐츠 서비스를 만듭니다.

Kokoji

So, let’s start! Tell us a bit about yourself. For example, name, profession, and personal interests. My name is Oleg Kokorin; I’m a CEO of Businessware Technologies (a custom software development company) and a machine learning engineer. My main interests are in teaching computers to see and write, diving, and traveling the world! I also frequently participate in Kaggle competitions. Interesting! What was your latest Hackernoon Top story about? My latest article -- an animal recognition app we at Businessware Technologies have developed for a startup. In this article, I describe how we went from nothing to a powerful computer vision system capable of detecting animals in real-time. was about Birdsy Do you usually write on similar topics? If not, what do you usually write about? Yes, my blog here on HackerNoon is focused on the topics of AI, machine learning, and computer vision, specifically on how to create an AI product from scratch, as it’s something I’ve had a lot of experience with. I also enjoy writing guides and other helpful content for AI startups as I feel like I have amassed enough experience and knowledge worth sharing with upcoming entrepreneurs. Great! What is your usual writing routine like (if you have one?) I don’t really have a routine, but I always talk to my team from our and ML department as they are a bright bunch that can give a unique perspective as people who are every day deep in the trenches of AI development. This can either give me ideas for new articles or give me data for an article I’m already working on. AI For example, for my latest article, I have interviewed our top machine learning engineer who had hands-on experience with developing Birdsy. This made the article a lot more interesting, as personal experiences often are way more interesting than technical details. Being a writer in tech can be a challenge. It’s not often our main role, but an addition to another one. What is the biggest challenge you have when it comes to writing? Foor me, one of the biggest challenges when it comes to writing is making my articles engaging and interesting to read. We at Businessware Technologies have worked on so many interesting projects that have provided me with a lot of insights that I would love to share, but it’s often challenging to relay dry technical mumbo jumbo into an engaging text worth reading. I hope to get good enough with my writing that even the most tech-heavy text would be interesting to read for the average user. What is the next thing you hope to achieve in your career? I’m always looking forward to new projects with our clients, but at this moment, I’m excited to work more on predictive AI projects, i.e., artificial intelligence systems capable of predicting outcomes based on previous data. I’ve been focusing on computer vision systems for a good while; I think it’s about time to expand my horizons. Wow, that’s admirable. Now, something more casual: What is your guilty pleasure of choice? I enjoy a good show! Silicon Valley is at the top, for sure. Do you have a non-tech-related hobby? If yes, what is it? I enjoy diving and snorkeling! Recently-ish, I went to the Maldives and got to go snorkeling in the beautiful waters there. There’s nothing like swimming underwater with local marine life! What can the Hacker Noon community expect to read from you next? I’m planning to write about how we developed an AI for detecting technical drawings. Stay tuned! What’s your opinion on HackerNoon as a platform for writers? So far, it’s been great! I especially like the editor; it makes it very easy to format and edit an article. Thanks for taking time to join our “ ” series. It was a pleasure. Do you have any closing words? Meet the Writer Thank you for this opportunity!

Meet the Writer: HackerNoon's Contributor Oleg Kokorin, CEO of Businessware Technologies

PIECHAIN Demo: Exploring a Practical Blockchain Interoperability Framework

পাইচেন ডেমো: একটি ব্যবহারিক ব্লকচেইন ইন্টারঅপারেবিলিটি ফ্রেমওয়ার্ক অন্বেষণ করা

PIECHAIN-Demo: Erkundung eines praktischen Blockchain-Interoperabilitätsrahmens

Demostración de PIECHAIN: Explorando un marco práctico de interoperabilidad de Blockchain

Démo PIECHAIN : Explorer un cadre pratique d'interopérabilité de la blockchain

PIECHAIN डेमो: एक व्यावहारिक ब्लॉकचेन इंटरऑपरेबिलिटी फ्रेमवर्क की खोज

PIECHAIN デモ: 実用的なブロックチェーン相互運用性フレームワークの探究

PIECHAIN 데모: 실용적인 블록체인 상호 운용성 프레임워크 탐색

Demonstração PIECHAIN: Explorando uma estrutura prática de interoperabilidade Blockchain

Демонстрация PIECHAIN: изучение практической структуры взаимодействия блокчейнов

PIECHAIN Demosu: Pratik Bir Blockchain Birlikte Çalışabilirlik Çerçevesini Keşfetmek

Bản trình diễn PIECHAIN: Khám phá Khung tương tác Blockchain thực tế

PIECHAIN 演示：探索实用的区块链互操作性框架

**PALO ALTO, California, March 20th, 2024/Chainwire/--**Pilotfish, a prototype Sui extension, was able to grow smart contract execution throughput in direct proportion to the number of machines used. Mysten Labs, a team of leading distributed systems, programming languages, and cryptography experts and originators of the Sui Network, today announced a landmark achievement in scaling blockchain capacity. During testing and development in a Sui blockchain environment, Pilotfish, a prototype Sui extension, increased throughput by 8x when backed by 8 machines, successfully illustrating the possibility of linear scaling. Notably, the latency per transaction decreased as more machines were added during the test, proving the viability of linear horizontal scaling for low latency blockchain transactions for the first time ever on any blockchain. This proof-of-concept, was formally presented as an academic preprint titled, " ," and initially shared on January 29, 2024 as an article on the open access archive arXiv. Pilotfish: Distributed Transaction Execution for Lazy Blockchains “Pilotfish is a game changer as the blockchain industry seeks to achieve the same dynamic capacity increase currently possible only with centralized server technologies,” said Lefteris Kokoris-Kogias, one of the authors of the paper and researcher at Mysten Labs. "Pilotfish has the potential to push throughput well beyond what is currently at the forefront of high-performance blockchains, enabling transaction throughput levels no other blockchain existing today can achieve." Sui leads the industry in throughput by leveraging multiple cores for parallel transaction execution within individual validators. Using this approach, within months following mainnet, Sui reached 65.8 million transaction executions in a single day—the most of any blockchain ever. However, a single machine can only incorporate a finite number of cores, so throughput gains via this strategy are limited. Moreover, the machines capable of accommodating large numbers of cores are expensive to buy and use. The breakthrough solution that Pilotfish offers enables a single validator to employ multiple servers simultaneously. This new technology not only makes elastic capacity possible, but its auto-scaling architecture, akin to centralized solutions like AWS or Heroku, allows the use of smaller, cheaper servers available on demand. Nearly all other blockchains aiming to scale use some form of batching—grouping transactions together to be added all at once. The downside of this method is significant additional latency. Instead of transactions moving directly to finality, they are delayed waiting for others to be batched for execution with them. Unlike these other scaling solutions, Pilotfish achieves potentially linear throughput scaling without increasing latency. In the simplest terms, with the horizontal scaling enabled by Pilotfish, simply by engaging the requisite number of machines, a validator can adapt its throughput and resource use to the use case at hand. Alberto Sonnino, a research scientist at Mysten Labs, said, "Until now the only option to increase a validator’s capacity has been to upgrade to a more powerful machine. Pilotfish removes that constraint, making possible horizontal scaling by simply adding servers and without delaying execution. With Pilotfish, the question is no longer whether a level of throughput is possible, only how many servers it will require." Pilotfish achieves its remarkable scalability by enabling a single validator to execute transactions across multiple machines. Specifically, Pilotfish partitions the validation function into three key roles: (1) the Primary, which handles sequencing transactions using a high-throughput consensus, (2) SequencingWorkers (SWs), which store transactions and dispatch them for execution; and (3) ExecutionWorkers (EWs), which store the blockchain state and execute transactions received from the SWs. Different components may reside on multiple different machines to increase the capacity of the system with more precision. In experimental results, Pilotfish has already achieved remarkable results in terms of linear scaling, low latency, and support for various workloads. In testing, Pilotfish reduced network response times and maintained a latency envelope below 20ms. Future plans include enhancements like multiple SequencingWorkers, shard replication, and ultra-fast networking. Contact Mysten Labs lexi.wangler@mystenlabs.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here.

Mysten Labs Technology Prototype On Sui Provides First Proof Of Elastic Blockchain Scaling

Innovating Identity Verification: An Introduction to Proof of Personhood (PoP) Protocols

In Papers, Please, a popular video game, you play the role of a customs officer checking documents according to increasingly complex rules. The main mechanic of the game is checking documents for compliance with all standards, such as the correct date and place of issue, compliance with the person’s first and last name on all documents, visa validity, whether the person is on “black” lists, and more. The game has attracted thousands of users around the world with its original style and unusual game mechanics, but for some people, this game would seem like a real nightmare because in real life, in their real workplace, they do the same thing. What Is KYC and Who Needs It KYC stands for Know Your Customer — a set of practices targeted towards obtaining as much information about a new customer as possible to weed out unreliable customers and reduce fraud. A comprehensive background check of a client, including their documents, bank history, and even activity on social networks, is included in the standard set of KYC practices aimed at reducing risks, which can be different for each business. Thus, a bank employee is forced to check not only the documents provided by the client but also interact with authorities in order to find out how respectable the person is and how likely they are to pay a loan back, and analyze their behavioral characteristics, for example, their behavior on social media. Online service managers, like online marketplaces and booking sites, on the other hand, do not need to analyze the user’s credit history, but it is important for them to know that their client does not use various loopholes, for example, does not register several times in a row using a temporary email to take advantage of a promo code, or does not register another account in order to leave negative feedback in order to lower the rating of the service. For some businesses, for example, car sharing and any Uber-like services, it is important to know who exactly uses their services (or provides them). For example, Uber needs to know who was driving the car when it got into an accident. Crypto exchanges, despite rejecting almost any regulations aimed at them, are forced to turn to KYC practices to identify clients due to local and global legal pressure. For example, to register on Binance, you not only need to enter your name and date of birth but also submit a photo of your ID, upload a selfie, and go through face verification using a webcam. Online dating sites and apps also strive to implement as many KYC practices as possible into the account registration process in order to avoid catfishing because this phenomenon can greatly reduce the number of users and undermine the reputation of the service. How It Usually Works KYC measures, in fact, are often very simple, both for the client and for the one who implements them. For example, verifying an email address via a message with a code or a phone number via a text message are the simplest (and most unreliable) examples of KYC. These measures are easily abused: everyone knows about temporary email services that are used to create multiple accounts in one service, which can be very useful for endlessly extending the free period of using the service, for example. These measures are not enough for businesses that value their user base. Let's talk about several options for making user verification more complex at the registration stage. Step 1: Are you human? The simplest step is the implementation of Captcha, which at least partially solves the problem with the mass registration of bots. This is very simple, cheap, and quite effective, with the reduction of bots noticeable almost instantly. However, Captcha alone cannot solve the problem of multi-accounts, a situation where one person registers in the system several times with malicious intent. Step 2: Determine IP Checking your clients' IPs is the next step in weeding out unreliable users. If different users regularly log into your service from the same IP, this may raise doubts about their integrity and purpose for using the service. In addition, detecting IPs of popular VPN services can also help reduce bots and unreliable customers. Step 3: Email and phone number verification Here, we are no longer talking about checking the user’s access to the specified phone number and email address but a more in-depth data verification. The validity of the email domain, the email registration date, and the connection with accounts on social networks are often checked. A phone number can also provide quite a lot of useful data, such as cellular operator, country, and connection to social networks. This way, you can identify those who created an email only for registration or use a burner phone. Step 4: Papers, please! This stage of improving verification significantly increases the number of steps during registration for the user, but it also increases the reliability of the service and significantly reduces the number of unscrupulous users within it. The person is asked to provide an identification document and enter their details when registering. Checking the validity of IDs is generally a simple task, as data on valid and invalid documents is available online. Step 5: Show your face There is still room for loopholes in the previous step. Nothing prevents a user who has been banned from a certain service but who ardently wants to continue using it from borrowing a passport from an unsuspecting grandmother and entering her information when registering a second account. This situation is extremely unpleasant for services seeking to maintain a healthy user base, and therefore, there needs to be an additional step: facial recognition. The most convenient way to implement facial recognition is in a mobile application. When registering, the user enters their ID information, takes a photo of the ID, and then the app asks them to make faces at the camera, distracting them from the facial recognition process, during which the following is performed: Comparison of the person’s face with their photo from the ID to determine whether the document belongs to them, Comparison with the database of faces of already registered users so that a person does not register several times using different documents, like a driver’s license instead of an ID, Check for liveness or whether a real person is in front of a camera and not their photo or a prerecorded video. Smart KYC: Real-life application and development Having had an opportunity to work on multiple AI-powered KYC systems for both web and mobile applications, I have a good understanding of how theory applies to practice. All of the steps outlined above are used in real-life applications, from simple email verification to facial recognition. While there are many ways to implement these practices in applications, I’ve compiled techniques and technologies behind smart KYC in modern systems. ID Recognition There are many different tools for document detection, all of which are geared toward different document types and different use cases. Most of them are paid, too, which is not ideal for a KYC system since the cost for each onboarding client would quickly pile up. One of the tools best suited for this application is Google Vision API, which detects text in any language and creates a bounding box around each word. Google Vision allows up to 1000 documents to be processed monthly for free, which should cover a decent chunk of new users for many services. To begin with, using the user-provided passport photo, we use OpenCV to determine the outline of the passport. This selection may not always go smoothly, for example, if the photo was taken against a colorful background or there are other objects nearby. In this case, we find the largest contour and believe that this is the passport. The resulting image is passed on to Google Vision, which identifies and highlights each word separately. After Google Vision detects each word, you can search for keywords like “Name”, “Surname”, “Date of Birth”, “Place Of Birth”, etc. Each passport type, which equals to country of origin, has its own config file with the following information: Keywords, or field titles, Spaces and line spacing Concerning spaces Do you think that countries differ from each other in culture, history, and their people? No, they differ in the size of spaces and line spacing in the passports of their citizens. For example, Romanian passports have short spaces, while Swedish passports have long spaces. The size of spaces is important to find the necessary information: after a keyword, in this case, field name, is found, the system looks for its value or field content, which can be either to the right of the keyword or below. Knowing the size of spaces and line spacing of a certain type of passport, we know how much to move to the right or down to find the value of a keyword. The size of spaces and intervals is not absolute but relative (depending on the size of the image). Facial recognition Another important aspect of smart KYC is facial recognition. With today’s abundance of excellent facial detection algorithms, there’s no need to reinvent the wheel and create your own bespoke solution, it’s much more cost-efficient to use a pretrained model for this job. For example, FaceNet is a good solution for the task of comparing two face photos and determining if it’s the same person or not. A simple script can extract a photo from an ID, while FaceNet (or any facial recognition model of your choice) analyzes the user's face using a smartphone camera and compares the two images to determine if the ID actually belongs to the user. Liveness detection Another important task when it comes to KYC is liveness detection, which is a set of techniques aimed at determining whether a source is a live human or a fake presentation. This is a vital step to eliminating multi-account fraud and preventing identity fraud. Liveness checks usually utilize motion detection and face detection algorithms to confirm the user’s identity. The easiest — and most effective — way to implement liveness detection is by prompting a user to perform a series of simple movements, like turning the head left or right, winking, smiling, etc. Again, there are several algorithms and models for movement detection on the market; it's a matter of choosing the right one for the task at hand. I recommend Google ML Kit as it's very easy to use and implement into an app. Summing Up Modern AI-based services and technologies make it possible to quickly and inexpensively implement a system that will eliminate account and identity fraud and improve KYC efforts. Well-implemented KYC measures are easy for the user to follow and complete and are effective enough to weed out users with bad intentions.

Papers, Please! - Know Your Customer With AI

have been all the rage for the past decade, but in the last year or two, the hype around AI has grown to unimaginable levels. Artificial intelligence and machine learning While artificial intelligence is capable of performing a wide variety of tasks, like recognizing and tracking objects in photos and videos, generating human-like text, extracting information and extrapolating from large amounts of data, and much more, it’s not the magic pill for every ailment. AI systems, while being widespread, may still not be the perfect fit for solving a given problem. One of the main reasons behind companies rejecting AI is cost. Artificial systems may seem omnipresent, thus inducing the feeling of them being easy and affordable to create, but this is far from the truth. Even the most simple AI-powered app of the system requires a significant time and money investment, and it’s not always obvious just how much of a commitment artificial systems really are. In this article, I aim to help businesses assess the cost of AI development more accurately as well as evaluate whether AI is truly going to save money in the long run. The Cost Of AI Development One of the main misconceptions about AI is that due to its popularity, it’s affordable and easily attainable. Artificial intelligence software, despite being widely used in many business operations, is complex to develop and takes a lot of time and, consequently, money. It’s crucial to calculate as precisely as possible the cost of AI development beforehand to avoid wasting time looking for an AI development team and disappointment when the total project cost doesn’t fit your budget. Calculating the development project is not the main step in the process of assessing the cost of AI development. In my opinion, calculating how much money it takes to complete a task you’re looking to automate now, using manual labor, is far more important. On the surface, it may seem like automating a certain task is the most sensible approach, when in reality, implementing an AI system to replace human labor is more often than not too expensive and not as effective compared to human labor. Calculate The Current Cost Before looking for AI developers, you must understand your business processes and have a clear look at how much it costs to complete a task you are looking to automate with AI. For example, a complex AI surveillance system that counts how many clients visit a store in a day may seem like a good idea on the surface: the counting process is automatic, and AI can not only count the people in the store but also track their purchases or general behavior. Falling for the AI hype, a business owner may consider getting such a system developed to get insights into their business operations. However, the cost of developing a real-time video processing AI for object detection is likely to be far more expensive, especially if your business is on a smaller side, than hiring an additional employee who will complete the same tasks for much less money. On the other hand, certain tasks may require a lot more human labor. For example, a company working with a lot of paperwork may find itself with 50+ employees sorting through hundreds of paper documents daily, looking for relevant information, and punching it into the systems manually. The cost of labor for over 50 employees, in the long run, is considerably more than the cost of developing an AI-powered OCR system for automatic document processing. Even with the upfront cost of system development, the AI solution remains the more sensible option for the business money-wise. This step may seem obvious, but many business owners seem to skip it entirely, falling for the AI hype. AI remains a complex technology that is rather expensive to develop and requires lots of data, lots of testing, and lots of time. While AI is superior to humans in completing certain tasks and may seem like the ideal choice for a certain business scenario, it may not make sense economically even in the long run. Calculating the cost of labor a certain task requires is the first step in developing any AI project for a business. Estimate Project Scope Some AI systems are smaller and less complex than others, and understanding where your project lies on this spectrum is the best way to get a precise project cost estimation. Here are a few questions to ask yourself to understand the size of the project at hand: Define The Project Goal Define a precise goal of the AI system, whether it’s looking for a specific piece of information within a document, detecting forest fires, or facial recognition for an onboarding process. It’s better to define the goal as precisely as possible, e.g. “an AI for detecting if a document has been signed and by who” instead of “an AI for processing documents automatically”. Define The AI Type Object recognition and classification, OCR, natural language processing, data analysis are all different in what type of AI models are used and how complex they are. Defining what type of an AI model your project requires will help with estimating the total project complexity and, therefore, its cost. Assess The Dataset A dataset is the cornerstone of any AI project. A dataset is used for training and testing machine learning models, and is often one of the most time-consuming and challenging parts of a project. Datasets are not small collections of images: to get accurate results, you need to include hundreds, better thousands of images, videos, audio recordings, scanned documents, etc. Estimate if you already have a dataset, for example, an archive of scanned paper documents or videos from surveillance cameras, and if not, estimate the time it would take to gather the needed data. Calculate Project Cost A precise cost calculation of any AI project can only be done by a professional AI development company that will assess your project and your business needs, test and choose ML models to get the best accuracy and provide you with a breakdown of different development stages. There are certain guidelines when it comes to AI project costs. Here’s how much time a project requires depending on its complexity: A tiny AI project with very limited functionality and a predictive AI, e.g. a model that identifies patterns in past events and makes predictions about future events, takes , at least 100 hours to develop A small AI project, including object recognition and classification, , starts at 2000 to 2500 hours Larger AI projects can easily . span across multiple years The hourly rate of any project is the main metric when it comes to calculating how much money the project will require. By multiplying this number by the amount of hours a project takes, you can get an assessment of the project cost. The rate of AI developers varies greatly based on the region, the developer’s qualification and their experience. Make sure to research the rates of AI projects in your area (or in the area where you are planning to outsource the development) to get a more accurate cost assessment for your project. For the sake of example, I will calculate the approximate AI project cost based on an average AI developer rate. According to Salary.com, the average hourly rate of an AI developer is around $50. Let’s look at the hours I’ve listed earlier for different AI projects. For example, a very small AI project with a simple pattern detection and event prediction model takes at least 100 hours. Multiplying this number by the average developer’s rate, we get $5000 — the approximate cost of this type of project. Performing this very simple calculation with the rest of the hourly estimates I’ve listed earlier results in the following numbers: A very small AI project with a simple pattern detection and event prediction model costs at least $5000, A small AI project with an object detection model starts at $100.000, A large AI project with multiple starts at $200.000. There are rough estimates, but they can be a start at assessing the cost of your AI project. Keep in mind that the hourly rate of AI developers depends in the region and county and can vary, significantly affecting the total cost of development. Additional costs, like the hourly rate of a project manager, servers, etc. also add up to the total project cost. Assess Additional Costs While additional costs are not the main chunk of the total investment, they are important to keep in mind when assessing whether an AI system is worth it for your business. A lot of custom software runs in-cloud, meaning you need to pay a monthly or yearly fee for renting servers. These costs are usually small enough to not have a large impact on the overall cost of development, but AI systems are different in this regard. AI systems require servers with powerful GPUs to run efficiently, and those tend to be more expensive, thus increasing the cost of monthly system upkeep. If you’re looking to develop a complex AI system or an AI system that will be used by dozens of employees, you need to account for tech support to guide users through the system and fix any bugs or malfunctions a system may experience. Another important cost to consider is the cost of additional development. During the system operation you may develop a need for additional functionality. For example, you may need to start processing new document types, or detect more objects on a video. These additions can be costly, so you need to keep them in mind when calculating the total project budget. The Cost Of AI vs Human Labor After you roughly estimate the cost of the AI system in question, compare it to the money you now spend on completing the same task. You may be surprised, but more often than not the price of human labor is much cheaper than the price of a custom-made AI system. Even with all the hype around AI, it’s important to keep your head cool and hype-free when looking into having an AI developed. Tempting as it is, implementing AI is not always a sensible idea due to economics.

AI Is More Expensive Than You Think

AI আপনার ধারণার চেয়ে বেশি ব্যয়বহুল

KI ist teurer als Sie denken

La IA es más cara de lo que crees

L’IA coûte plus cher que vous ne le pensez

AI आपके विचार से कहीं अधिक महंगा है

AI は思っているよりも高価です

AI는 생각보다 비싸다

IA é mais cara do que você pensa

ИИ дороже, чем вы думаете

Yapay Zeka Düşündüğünüzden Daha Pahalı

AI đắt hơn bạn nghĩ

人工智能比你想象的更昂贵

In this article, I will dive into a complex world of complex document recognition using AI and OCR. Document recognition nowadays is not a complex task. Modern OCR solutions are able to detect both typed and written text in many languages. One can find dedicated solutions for the detection of specific documents like passports and driver’s licenses. But where out-of-the-box tends to struggle is when a document includes special symbols or tilted text. AI Technical drawings are among the ‘trouble children’ that cause ready-made OCR solutions to struggle: they are nothing but a collection of weird symbols and weirdly placed text. Having worked on an AI solution for technical drawing recognition, I have insights into the world of modern OCR that I will share in this article. Why OCR is bad for OCR The ‘digital first’ approach, at the forefront of many businesses, has motivated many to convert physical documents into a digital format. This process usually involves the implementation of OCR — optical character recognition — which converts physical documents into PDF files. Morel OCR tools are capable of recognizing more than just text. In many cases, OCR tools can detect special symbols, written text, signatures, images, and more. Many of these tools come ready to use: all you need to do is install the tool (or, if you are working on a custom solution, use an API) to scan the documents in question. Despite all this, OCR tools have certain limitations. They don’t work well for irregular text, also called wild text, like low-quality scanned documents with no predefined structure, car license plates, text on advertisement billboards, etc. Low-quality scans The quality of text recognition depends highly on the quality of the document itself. Warping, scratches, faded ink, and more have a detrimental effect on the recognition quality. Symbol mixups Even the best OCR tools have trouble distinguishing between certain similar-looking letters and numbers, like ‘3’ and ‘8’ or ‘O’ and ‘D.’ The very challenges OCR is supposed to solve often become the stumbling block of document digitization. Special symbols Documents that feature any special symbols, from letters specific to a certain language to symbols denominating certain objects, like symbols used in technical drawings, e.g., diameter ‘Ø,’ square ‘□.’ AI to the rescue Using artificial intelligence, OCR tools can be improved and augmented to better handle complex documents, and often even replaced by a custom . AI neural network Model-based OCR, or intelligent OCR, is the result of using deep learning for text document recognition. Neural networks can be trained to recognize text regular OCR tools have trouble with. Intelligent OCR provides superior text recognition results in document recognition applications by improving recognition speed and reducing errors. Recognition of complex documents Despite the widespread digitization, some paperwork remains offline. This usually applies to complex documents that are impossible to digitize due to their complex layouts, the use of special symbols, and unconventional formatting. Technical drawings are the perfect example of a complex document: their layouts change from one document to another; they include a bunch of symbols specific to technical drawings only, and the text is often formatted in odd ways. All of the above makes technical drawings the perfect candidate for model-based OCR. While working on a similar project, I’ve developed an understanding of the best strategies to apply when working on digitizing technical drawings. I have had experience with working on an AI for floor plan detection, so that’s what I’ll be using as an example. I’ve broken the process down into sections, as this is exactly how one should approach the development of AI-based OCR solutions for complex document recognition. Stage 1: Detection of text Recognition of plain text is the most simple part of this entire ordeal. When it comes to technical drawings, plain text is used to specify the drawing type, dimensions, floor plan type, building type, etc. While the detection of plain text is a simple task, detecting text on a technical drawing is far more complex. The text can come in a variety of fonts, sizes, and colors, can be rotated or upside down, and contains special symbols. Ready-made OCR software like iText and OCRSpace can detect simple text with high accuracy, but they fail spectacularly when it comes to technical drawings (or any other complex document, for that matter). For example, these tools struggle to detect rotated text. Most OCR tools can be fine-tuned to handle problematic text better. . The best approach to recognizing complex text is to use multiple fine-tuned OCR tools along with a balancer that compares the results of each tool and chooses the one that produces the most accurate results Another benefit of using fine-tuned OCR software is the increase in recognition speed. By fine-tuning these tools alone, we’ve seen a 200 times decrease in document processing speed.If you add an OCR engine into the equation, like Tesseract, the text recognition quality can be increased up to 99.9%. Stage 2: Recognition of special symbols Each technical drawing includes special symbols of some sort. In the case of floor plan technical drawings, the documents include symbols designating doors, windows, electrical outlets, etc. These symbols, or labels, look like geometric figures with text inside. They can be difficult to distinguish from their surroundings due to their shape, which blends in perfectly with the rest of the drawing. In addition, there can be multiple labels representing the same object due to inconsistencies in document design. Pre-trained computer vision solutions, like OpenCV libraries for symbol detection, work best with photographs of real-life objects. Technical drawings are quite a bit different: they are almost always in black and white and mostly consist of geometric shapes. We’ve tested multiple OpenCV libraries, each of which resulted in albeit different, yet insufficiently low recognition quality. Unless you develop your own neural network from scratch, any pre-trained computer vision model needs to be built upon to achieve decent recognition quality. One of the main problems with using is the amount of false positive results they produce. Technical drawings consist of simple geometric shapes, but so do special symbols and labels, which results in CV models detecting random parts of the drawings as labels. pre-trained CV models The best way of mitigating this issue is to implement deep learning to detect false positive results and remove them from the final detection results. Stage 3: Spreadsheets Technical drawings often include large spreadsheets with merged cells and complex structures stretching across multiple pages. While spreadsheets are generally easy to detect, the complex nature of these spreadsheets makes them difficult to crack. Going a custom software route is the best way to achieve satisfactory results. Here’s how we’ve done it: Recognition of text in a spreadsheet Solutions like Amazon Textract work very well and can extract text with very high accuracy as long as the document scan is of high quality. Documents with 300 DPI result in 100% recognition accuracy and 100 DPI results in ~90% accuracy. Recognition of spreadsheet structure First, you need to detect the spreadsheet structure by detecting vertical and horizontal lines. Using OpenCV, create a binary matrix by converting the document into black and white, defining its threshold in a way that results in all horizontal and vertical lines being one and the rest — a zero. The binary matrix will then contain the spreadsheet structure. Using the extracted text and spreadsheet structure, the spreadsheet itself can be extracted in an editable format like Excel. Summing Up Digitizing any complex document comes with its own set of problems. The best approach to solving them is to approach them one by one, researching the best tools for the job, testing them, and comparing results. The approaches I’ve described work on any document type despite its type, as individual challenges can be similar despite the document type being completely different. For example, I have experience in working on a passport detection solution where the text recognition challenges were very similar, and we’ve used some of the same techniques. Knowing your OCR tools, being well-versed in coding neural networks and having decent experience in the field of custom AI development will help overcome any document digitization challenges.

Complex Document Recognition: OCR Doesn’t Work and Here’s How You Fix It

Usually, image recognition is done using computer vision and machine learning. The neural network trains on a set of images from which it learns to recognize certain objects in an image. As good as are, they are not always the best choice for the job. neural networks Certain restrictions, like the inability to retrain the model when new object classes are added or weak hardware, make it impossible to use traditional methods of image recognition. In this article, I will share a method for image recognition that doesn’t involve neural networks and share my experience with creating a mobile app based on this approach. Project requirements and restrictions The reason for ditching neural networks and searching for a different way of recognizing objects is project restrictions. In my case, the app was created for an art museum with the intent of augmenting the visitors’ experience: guests would point their smartphone camera at a painting and receive information about the piece and the artist. One of the main restrictions of this project was the ability to add new art pieces to the dataset without the need for model retraining, as well as quick recognition times of less than 1 second. Based on the project requirements, we had to refrain from using neural networks and go for a instead. classic algorithm Classic algorithms for object detection Classic algorithms have certain advantages over neural networks: They don’t require training They can work using much weaker hardware The object recognition is done by searching for key points: Key points are points of spatial location that define whatever stands out about an image. Mathematically speaking, a key point is a point of high contrast, a point of high gradient value. For example, key points of a chess field would be points where black squares meet white ones. A totally white image won’t have any key points as there’s no change in color within the image, whereas if we add another color, the key point will be the transition between the white background and the new color. Back to the app in question, the art is analyzed by searching for key points. Key points have coordinates that are determined by searching for points with maximum contrast. Key point detection using OpenCV OpenCV has a — a key point detector and descriptor. Using KAZE, we can search for key points in an image and generate a feature vector for each point. KAZE class The algorithm reads the vector around each key point in all directions and generates a number value that describes the key point. Based on these values, we can compare the key points by measuring the distance between vectors. Key point image detection process Here’s how the process of image recognition works when using the key point approach: The app receives an input image It analyzes the image and locates its key points The key points are then compared to those in a dataset The dataset image which has the most key points in common with the input image is determined as the winner , and image recognition is done only by using a mathematical approach. The algorithm requires no training Testing the algorithm It’s time to test the idea in practice and to do that, we have . All you need to do is send an image, and the system gets back to you with recognition results. created a Telegram bot Based on these tests, we have seen that this approach not only works but is the most optimal one, given the restrictions of the project. The initial recognition accuracy was around 60%, which definitely needed an improvement along with recognition speed on mobile devices. Improving detection accuracy One approach to increasing recognition quality is the collection of key points from multiple images of the same object taken from different perspectives. This way, we would have more information about the object, thereby increasing recognition accuracy. However, this approach has a catch: due to the large number of duplicate key points, point scoring started to work incorrectly. The easiest way to explain what went wrong is with a simple example. Let’s say we have a dataset containing photos of two people, Garry and Mary. We have two photos of Garry and ten photos of Mary. The algorithm determines all of Garry’s key points and puts them in one group, all of Mary’s key points — into another. Now, let’s show a photo of Garry to the algorithm. Garry and Mary are both humans and have a lot of similar features (eyes, nose, mouth, ears, etc.), and many of their key points will be similar as well. The algorithm will find seven similar key points in the “Mary” group and only 2 in the “Garry” group, thus making an incorrect assumption that the photo depicts Mary, which is incorrect. One of the groups can win based only on a larger number of photos in a dataset. We have solved this issue by replacing groups of similar key points with a centroid — an average of the feature vector. Doing this increased recognition quality from 60% to 90%. Additional increases in recognition quality can be made by defining requirements for images in a dataset. Increasing the speed of determining distance Object detection based on key points comes down to assessing the similarity between them, for which you need to calculate the distance between the key point’s descriptors. A Naive Bayes algorithm was too slow because, for each point on the test image, it needed to calculate the distances to all points in the dataset. Due to the sheer amount of points, this approach was very slow. To speed things up, we have replaced that algorithm with HNSW — an algorithm for approximate search of nearest neighbors — which builds a hierarchical space graph. [3] Before the implementation of HNSW, the recognition took multiple seconds; after the implementation — 1 to 3 fps. Approach restrictions The key point approach works perfectly within the constraints of this project. However, it’s not always an ideal choice and has a set of limitations. Lack of key points for certain objects First of all, as I’ve mentioned earlier, not all objects have key points. Solid color or very low contrast objects can’t be detected or recognized using a key point approach. Accuracy score complications Secondly, it’s not easy to give an accurate and definitive assessment of recognition confidence. The confidence score is calculated by counting the matching key points for each image class. Every class has its own number of points; for example, class 1 has 3 points, class 2 has 4 points, etc. The class with the most points determines the recognition result. When there are , the entire number of points goes into a denominator, and the winner’s points go into the numerator. For example, the sum of all points of all classes is 100. The winner has 10 points, and the rest of the classes have 1 point each, but if we divide 10 by 100 the confidence score will be very low. a lot of classes in a dataset False positives Thirdly, it’s difficult to ignore an input image that’s not present in a dataset. The algorithm will always find the closest similar image in a dataset, even if it has just one similar key point.

Ditch the AI for a Second: Image Recognition Without Neural Networks

KOKOJI

kokozi.house

-- employees

Since n.d.

COMPANY RANKING

KOKOJI

EVERGREEN INDEX #5587

Representing this company? Click here to claim and customize this page!

Kokoji WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

KOKOJI

kokozi.house

-- employees

Since n.d.

COMPANY RANKING

KOKOJI

EVERGREEN INDEX #5587

RELATED COMPANIESRANK

STORIES ABOUT KOKOJI

Representing this company? Click here to claim and customize this page!

Kokoji WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!