Kokoji

kokoji

So, let’s start! Tell us a bit about yourself. For example, name, profession, and personal interests. My name is Oleg Kokorin; I’m a CEO of Businessware Technologies (a custom software development company) and a machine learning engineer. My main interests are in teaching computers to see and write, diving, and traveling the world! I also frequently participate in Kaggle competitions. Interesting! What was your latest Hackernoon Top story about? My latest article -- an animal recognition app we at Businessware Technologies have developed for a startup. In this article, I describe how we went from nothing to a powerful computer vision system capable of detecting animals in real-time. was about Birdsy Do you usually write on similar topics? If not, what do you usually write about? Yes, my blog here on HackerNoon is focused on the topics of AI, machine learning, and computer vision, specifically on how to create an AI product from scratch, as it’s something I’ve had a lot of experience with. I also enjoy writing guides and other helpful content for AI startups as I feel like I have amassed enough experience and knowledge worth sharing with upcoming entrepreneurs. Great! What is your usual writing routine like (if you have one?) I don’t really have a routine, but I always talk to my team from our and ML department as they are a bright bunch that can give a unique perspective as people who are every day deep in the trenches of AI development. This can either give me ideas for new articles or give me data for an article I’m already working on. AI For example, for my latest article, I have interviewed our top machine learning engineer who had hands-on experience with developing Birdsy. This made the article a lot more interesting, as personal experiences often are way more interesting than technical details. Being a writer in tech can be a challenge. It’s not often our main role, but an addition to another one. What is the biggest challenge you have when it comes to writing? Foor me, one of the biggest challenges when it comes to writing is making my articles engaging and interesting to read. We at Businessware Technologies have worked on so many interesting projects that have provided me with a lot of insights that I would love to share, but it’s often challenging to relay dry technical mumbo jumbo into an engaging text worth reading. I hope to get good enough with my writing that even the most tech-heavy text would be interesting to read for the average user. What is the next thing you hope to achieve in your career? I’m always looking forward to new projects with our clients, but at this moment, I’m excited to work more on predictive AI projects, i.e., artificial intelligence systems capable of predicting outcomes based on previous data. I’ve been focusing on computer vision systems for a good while; I think it’s about time to expand my horizons. Wow, that’s admirable. Now, something more casual: What is your guilty pleasure of choice? I enjoy a good show! Silicon Valley is at the top, for sure. Do you have a non-tech-related hobby? If yes, what is it? I enjoy diving and snorkeling! Recently-ish, I went to the Maldives and got to go snorkeling in the beautiful waters there. There’s nothing like swimming underwater with local marine life! What can the Hacker Noon community expect to read from you next? I’m planning to write about how we developed an AI for detecting technical drawings. Stay tuned! What’s your opinion on HackerNoon as a platform for writers? So far, it’s been great! I especially like the editor; it makes it very easy to format and edit an article. Thanks for taking time to join our “ ” series. It was a pleasure. Do you have any closing words? Meet the Writer Thank you for this opportunity!

Meet the Writer: HackerNoon's Contributor Oleg Kokorin, CEO of Businessware Technologies

The Greedy Miner’s Dilemma: Flaws in DAG-Based Consensus Protocols

The Hypersonic Drone Market: How the Axis of Innovation Is Forming Around It

This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.
Authors:
(1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. Table of Links Abstract and Introduction
Grassroots Protocols
Grassroots Implementation of Grassroots Dissemination
Applications of Grassroots Dissemination Supporting Digital Sovereignty
Future Work & Discussion
References
A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems
B. All-to-All Dissemination is not Grassroots
C. Proofs References 1 Martin R Albrecht, Jorge Blasco, Rikke Bjerg Jensen, and Lenka Mareková. Mesh messaging in large-scale protests: Breaking bridgefy. In Cryptographers’ Track at the RSA Conference, pages 375–398. Springer, 2021. 2 Juan Benet. Ipfs-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561, 2014. 3 Sonja Buchegger, Doris Schiöberg, Le-Hung Vu, and Anwitaman Datta. Peerson: P2p social networking: early experiences and insights. In Proceedings of the Second ACM EuroSys Workshop on Social Network Systems, pages 46–52, 2009. 4 Vitalik Buterin et al. A next-generation smart contract and decentralized application platform. white paper, 3(37), 2014. 5 Manuel Castells. The city and the grassroots: a cross-cultural theory of urban social movements. Number 7. Univ of California Press, 1983. 6 Miguel Castro, Barbara Liskov, et al. Practical byzantine fault tolerance. In OsDI, volume 99, pages 173–186, 1999. 7 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Constructing scalable overlays for pub-sub with many topics. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, pages 109–118, 2007. 8 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Spidercast: a scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the 2007 inaugural international conference on Distributed event-based systems, pages 14–25, 2007. 9 Bram Cohen. Incentives build robustness in bittorrent. In Workshop on Economics of Peer-to-Peer systems, volume 6, pages 68–72. Berkeley, CA, USA, 2003. 10 Alan Demers, Karin Petersen, Mike Spreitzer, Doug Terry, Marvin Theimer, and Brent Welch. The bayou architecture: Support for data sharing among mobile users. In 1994 First Workshop on Mobile Computing Systems and Applications, pages 2–7. IEEE, 1994. 11 Bitcoin Developer. P2p network, Retrieved 20222. URL: https://developer.bitcoin.org/ devguide/p2p_network.html. 12 Ethereum Developer. go-ethereum/params/bootnodes.go, Retrieved 2023. URL: https://github.com/ethereum/go-ethereum/blob/master/params/bootnodes.go. 13 Adem Efe Gencer, Soumya Basu, Ittay Eyal, Robbert van Renesse, and Emin Gün Sirer. Decentralization in bitcoin and ethereum networks. In International Conference on Financial Cryptography and Data Security, pages 439–457. Springer, 2018. 14 Gabriele Goldacker. Digitale souveränität [white paper of kompetenzzentrum öffentliche it (öfit)]. retrieved april 20, 2021, 2017. 15 Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. All you need is dag, 2021. arXiv:2102.08325. 16 Idit Keidar, Oded Naor, and Ehud Shapiro. Cordial miners: A family of simple and efficient consensus protocols for every eventuality. arXiv preprint arXiv:2205.09174, 2022. 17 Antony Lewis. A gentle introduction to self-sovereign identity. 2017. URL: https://bitsonblocks.net/2017/05/17/a-gentle-introduction-to-self-sovereign-identity/. 18 Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2019. 19 Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani. Epidemic processes in complex networks. Reviews of modern physics, 87(3):925, 2015. 20 Julia Pohle and Thorsten Thiel. Digital sovereignty. Internet Policy Review: Journal on Internet Regulation, 9(4):1–19, 2020. 21 Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. Challenges in the decentralised web: The mastodon case. In Proceedings of the Internet Measurement Conference, pages 217–229, 2019. 22 Vinay Setty, Maarten van Steen, Roman Vitenberg, and Spyros Voulgaris. Poldercast: Fast, robust, and scalable architecture for p2p topic-based pub/sub. In ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 271–291. Springer, 2012. 23 Devavrat Shah et al. Gossip algorithms. Foundations and Trends® in Networking, 3(1):1–125, 2009. 24 Ehud Shapiro. Multiagent transition systems: Protocol-stack mathematics for distributed computing. arXiv preprint arXiv:2112.13650, 2021. 25 Ehud Shapiro. Sovereign personal cryptocurrencies: A grassroots foundation for a digital economy. arXiv preprint arXiv:2202.05619, 2022. 26 Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. Conflict-free replicated data types. In Symposium on Self-Stabilizing Systems, pages 386–400. Springer, 2011. 27 Robert Shostak, Marshall Pease, and Leslie Lamport. The byzantine generals problem. ACM Transactions on Programming Languages and Systems, 4(3):382–401, 1982. 28 Ion Stoica, Robert Morris, David Karger, M Frans Kaashoek, and Hari Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM computer communication review, 31(4):149–160, 2001. 29 Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. Hotstuff: Bft consensus with linearity and responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, pages 347–356, 2019. 30 Shoshana Zuboff. The age of surveillance capitalism: The fight for a human future at the new frontier of power: Barack Obama’s books of 2019. Profile books, 2019. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: Authors: (1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. Table of Links Abstract and Introduction Grassroots Protocols Grassroots Implementation of Grassroots Dissemination Applications of Grassroots Dissemination Supporting Digital Sovereignty Future Work & Discussion References A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems B. All-to-All Dissemination is not Grassroots C. Proofs Abstract and Introduction Abstract and Introduction Grassroots Protocols Grassroots Protocols Grassroots Implementation of Grassroots Dissemination Grassroots Implementation of Grassroots Dissemination Applications of Grassroots Dissemination Supporting Digital Sovereignty Applications of Grassroots Dissemination Supporting Digital Sovereignty Future Work & Discussion Future Work & Discussion References References A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems B. All-to-All Dissemination is not Grassroots B. All-to-All Dissemination is not Grassroots C. Proofs C. Proofs References 1 Martin R Albrecht, Jorge Blasco, Rikke Bjerg Jensen, and Lenka Mareková. Mesh messaging in large-scale protests: Breaking bridgefy. In Cryptographers’ Track at the RSA Conference, pages 375–398. Springer, 2021. 2 Juan Benet. Ipfs-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561, 2014. 3 Sonja Buchegger, Doris Schiöberg, Le-Hung Vu, and Anwitaman Datta. Peerson: P2p social networking: early experiences and insights. In Proceedings of the Second ACM EuroSys Workshop on Social Network Systems, pages 46–52, 2009. 4 Vitalik Buterin et al. A next-generation smart contract and decentralized application platform. white paper, 3(37), 2014. 5 Manuel Castells. The city and the grassroots: a cross-cultural theory of urban social movements. Number 7. Univ of California Press, 1983. 6 Miguel Castro, Barbara Liskov, et al. Practical byzantine fault tolerance. In OsDI, volume 99, pages 173–186, 1999. 7 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Constructing scalable overlays for pub-sub with many topics. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, pages 109–118, 2007. 8 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Spidercast: a scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the 2007 inaugural international conference on Distributed event-based systems, pages 14–25, 2007. 9 Bram Cohen. Incentives build robustness in bittorrent. In Workshop on Economics of Peer-to-Peer systems, volume 6, pages 68–72. Berkeley, CA, USA, 2003. 10 Alan Demers, Karin Petersen, Mike Spreitzer, Doug Terry, Marvin Theimer, and Brent Welch. The bayou architecture: Support for data sharing among mobile users. In 1994 First Workshop on Mobile Computing Systems and Applications, pages 2–7. IEEE, 1994. 11 Bitcoin Developer. P2p network, Retrieved 20222. URL: https://developer.bitcoin.org/ devguide/p2p_network.html. 12 Ethereum Developer. go-ethereum/params/bootnodes.go, Retrieved 2023. URL: https://github.com/ethereum/go-ethereum/blob/master/params/bootnodes.go. 13 Adem Efe Gencer, Soumya Basu, Ittay Eyal, Robbert van Renesse, and Emin Gün Sirer. Decentralization in bitcoin and ethereum networks. In International Conference on Financial Cryptography and Data Security, pages 439–457. Springer, 2018. 14 Gabriele Goldacker. Digitale souveränität [white paper of kompetenzzentrum öffentliche it (öfit)]. retrieved april 20, 2021, 2017. 15 Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. All you need is dag, 2021. arXiv:2102.08325. 16 Idit Keidar, Oded Naor, and Ehud Shapiro. Cordial miners: A family of simple and efficient consensus protocols for every eventuality. arXiv preprint arXiv:2205.09174, 2022. 17 Antony Lewis. A gentle introduction to self-sovereign identity. 2017. URL: https://bitsonblocks.net/2017/05/17/a-gentle-introduction-to-self-sovereign-identity/. 18 Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2019. 19 Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani. Epidemic processes in complex networks. Reviews of modern physics, 87(3):925, 2015. 20 Julia Pohle and Thorsten Thiel. Digital sovereignty. Internet Policy Review: Journal on Internet Regulation, 9(4):1–19, 2020. 21 Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. Challenges in the decentralised web: The mastodon case. In Proceedings of the Internet Measurement Conference, pages 217–229, 2019. 22 Vinay Setty, Maarten van Steen, Roman Vitenberg, and Spyros Voulgaris. Poldercast: Fast, robust, and scalable architecture for p2p topic-based pub/sub. In ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 271–291. Springer, 2012. 23 Devavrat Shah et al. Gossip algorithms. Foundations and Trends® in Networking, 3(1):1–125, 2009. 24 Ehud Shapiro. Multiagent transition systems: Protocol-stack mathematics for distributed computing. arXiv preprint arXiv:2112.13650, 2021. 25 Ehud Shapiro. Sovereign personal cryptocurrencies: A grassroots foundation for a digital economy. arXiv preprint arXiv:2202.05619, 2022. 26 Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. Conflict-free replicated data types. In Symposium on Self-Stabilizing Systems, pages 386–400. Springer, 2011. 27 Robert Shostak, Marshall Pease, and Leslie Lamport. The byzantine generals problem. ACM Transactions on Programming Languages and Systems, 4(3):382–401, 1982. 28 Ion Stoica, Robert Morris, David Karger, M Frans Kaashoek, and Hari Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM computer communication review, 31(4):149–160, 2001. 29 Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. Hotstuff: Bft consensus with linearity and responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, pages 347–356, 2019. 30 Shoshana Zuboff. The age of surveillance capitalism: The fight for a human future at the new frontier of power: Barack Obama’s books of 2019. Profile books, 2019.

Grassroots Distributed Systems for Digital Sovereignty: References

PIECHAIN Demo: Exploring a Practical Blockchain Interoperability Framework

পাইচেন ডেমো: একটি ব্যবহারিক ব্লকচেইন ইন্টারঅপারেবিলিটি ফ্রেমওয়ার্ক অন্বেষণ করা

PIECHAIN-Demo: Erkundung eines praktischen Blockchain-Interoperabilitätsrahmens

Demostración de PIECHAIN: Explorando un marco práctico de interoperabilidad de Blockchain

Démo PIECHAIN : Explorer un cadre pratique d'interopérabilité de la blockchain

PIECHAIN डेमो: एक व्यावहारिक ब्लॉकचेन इंटरऑपरेबिलिटी फ्रेमवर्क की खोज

PIECHAIN デモ: 実用的なブロックチェーン相互運用性フレームワークの探究

PIECHAIN 데모: 실용적인 블록체인 상호 운용성 프레임워크 탐색

Demonstração PIECHAIN: Explorando uma estrutura prática de interoperabilidade Blockchain

Демонстрация PIECHAIN: изучение практической структуры взаимодействия блокчейнов

PIECHAIN Demosu: Pratik Bir Blockchain Birlikte Çalışabilirlik Çerçevesini Keşfetmek

Bản trình diễn PIECHAIN: Khám phá Khung tương tác Blockchain thực tế

PIECHAIN 演示：探索实用的区块链互操作性框架

**PALO ALTO, California, March 20th, 2024/Chainwire/--**Pilotfish, a prototype Sui extension, was able to grow smart contract execution throughput in direct proportion to the number of machines used. Mysten Labs, a team of leading distributed systems, programming languages, and cryptography experts and originators of the Sui Network, today announced a landmark achievement in scaling blockchain capacity. During testing and development in a Sui blockchain environment, Pilotfish, a prototype Sui extension, increased throughput by 8x when backed by 8 machines, successfully illustrating the possibility of linear scaling. Notably, the latency per transaction decreased as more machines were added during the test, proving the viability of linear horizontal scaling for low latency blockchain transactions for the first time ever on any blockchain. This proof-of-concept, was formally presented as an academic preprint titled, " ," and initially shared on January 29, 2024 as an article on the open access archive arXiv. Pilotfish: Distributed Transaction Execution for Lazy Blockchains “Pilotfish is a game changer as the blockchain industry seeks to achieve the same dynamic capacity increase currently possible only with centralized server technologies,” said Lefteris Kokoris-Kogias, one of the authors of the paper and researcher at Mysten Labs. "Pilotfish has the potential to push throughput well beyond what is currently at the forefront of high-performance blockchains, enabling transaction throughput levels no other blockchain existing today can achieve." Sui leads the industry in throughput by leveraging multiple cores for parallel transaction execution within individual validators. Using this approach, within months following mainnet, Sui reached 65.8 million transaction executions in a single day—the most of any blockchain ever. However, a single machine can only incorporate a finite number of cores, so throughput gains via this strategy are limited. Moreover, the machines capable of accommodating large numbers of cores are expensive to buy and use. The breakthrough solution that Pilotfish offers enables a single validator to employ multiple servers simultaneously. This new technology not only makes elastic capacity possible, but its auto-scaling architecture, akin to centralized solutions like AWS or Heroku, allows the use of smaller, cheaper servers available on demand. Nearly all other blockchains aiming to scale use some form of batching—grouping transactions together to be added all at once. The downside of this method is significant additional latency. Instead of transactions moving directly to finality, they are delayed waiting for others to be batched for execution with them. Unlike these other scaling solutions, Pilotfish achieves potentially linear throughput scaling without increasing latency. In the simplest terms, with the horizontal scaling enabled by Pilotfish, simply by engaging the requisite number of machines, a validator can adapt its throughput and resource use to the use case at hand. Alberto Sonnino, a research scientist at Mysten Labs, said, "Until now the only option to increase a validator’s capacity has been to upgrade to a more powerful machine. Pilotfish removes that constraint, making possible horizontal scaling by simply adding servers and without delaying execution. With Pilotfish, the question is no longer whether a level of throughput is possible, only how many servers it will require." Pilotfish achieves its remarkable scalability by enabling a single validator to execute transactions across multiple machines. Specifically, Pilotfish partitions the validation function into three key roles: (1) the Primary, which handles sequencing transactions using a high-throughput consensus, (2) SequencingWorkers (SWs), which store transactions and dispatch them for execution; and (3) ExecutionWorkers (EWs), which store the blockchain state and execute transactions received from the SWs. Different components may reside on multiple different machines to increase the capacity of the system with more precision. In experimental results, Pilotfish has already achieved remarkable results in terms of linear scaling, low latency, and support for various workloads. In testing, Pilotfish reduced network response times and maintained a latency envelope below 20ms. Future plans include enhancements like multiple SequencingWorkers, shard replication, and ultra-fast networking. Contact Mysten Labs lexi.wangler@mystenlabs.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here.

Mysten Labs Technology Prototype On Sui Provides First Proof Of Elastic Blockchain Scaling

Innovating Identity Verification: An Introduction to Proof of Personhood (PoP) Protocols

How To Find The Best LLM For Your Project

Having worked on multiple technical drawing processing projects, it was a matter of time till an engineering drawing automation project came our way. What’s so special about engineering drawings, you ask? Geometric dimensioning and tolerancing annotations (GD&T) is your answer. These pesky labels often present a challenge when processing and extracting data from engineering drawings due to their position on a page and overall structure. But don’t fret — I’m here to share just how we managed to process GD&T annotations on engineering drawings with AI. Let’s start from the beginning though. Processing Unstructured Documents All digital documents can be separated into 2 types: structured and unstructured: Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. As you might have guessed, technical drawings are a classic example of an unstructured document: despite adhering to a strict set of standards, each drawing is different from another as they lack rigid structure. Coupled with a mix of typed and handwritten text data, special symbols, complex spreadsheets, and various annotations, technical drawings present a real challenge for extracting data automatically. The complex nature of technical drawings makes them the perfect candidate for AI data extraction. In fact, using neural models to detect and extract various data from the drawings is the only way to automate their processing. Modern computer vision models and a smart approach to product development can yield a powerful tool for quick processing of any technical drawing. A Problem With Ready-Made Tools One quick Google search will show you at least a couple of solutions for processing engineering drawings. Nearly all of them offer wide functionality and promise quick and accurate processing of complex data. At first glance, this may sound very promising: paying for a monthly subscription to process engineering drawings with high precision. In practice, however, things are often not as smooth. Ready-made tools often struggle to detect and process rotated elements as their algorithms are only trained to process the “common denominator”, which, in our case, is an engineering drawing with labels and annotations positioned horizontally. Therefore using a ready-made solution is only suitable for those whose drawings are relatively simple and include only standard data. Any deviation from the “common denominator” will present a challenge for a ready-made tool. Feature Extraction From Engineering Drawings This exact situation happened to one of our clients: solutions for processing engineering drawings on the market don’t answer the needs of processing complex or non-standard drawings, leaving them with poor data recognition results. GD&T annotations carry a lot of very important information that is vital to extract from the drawing for further processing, but their position on the page, in our case, them being positioned at an angle, throws a wrench into the process of analyzing drawings by a premade AI tool. This is where custom AI development comes into play: AI models trained to detect and extract information from your specific document can solve (almost) any challenge a ready-made tool struggles with. Here’s how we’ve solved one of the challenges of processing engineering drawings with custom AI model development — extraction of GD&T annotations placed at an angle. Step 1: Detecting Annotation Position The first step is to locate the position of annotations on the drawing. AI models can be trained to detect the location of annotations irrespective of their position or rotation angle. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Step 2: Detect Rotation Angle Here’s the important part: detecting how the annotation is rotated. The AI model needs to calculate the rotation angle and rotate the annotation to make it horizontal. The cut-out PNGs are then passed on for further processing: Step 3: Extracting Data From Annotations After all annotations are detected, rotated, and extracted from the drawing, they are run through a symbol detection engine. Tesseract is a good choice for this as it provides high accuracy of recognition and can work with multiline text and symbols of different heights. First, you need to find the exact area where the text is located to improve the symbol recognition process. I would recommend using OpenCV as it handles these tasks very well and is relatively easy to work with. Next, the detected text area is handed over to the OCR engine to extract all text and symbols. Step 4: Analyzing Data An array of letters, numbers, and symbols need to be interpreted to provide “digestible” data that humans — or a data management system — can understand and process. Detected symbols are separated into groups forming part dimensions, tolerances, fits, and radii. Step 5: Data Management Data extracted by an AI system needs to be extracted according to your needs: JSON files: Perfect for importing the data into existing software,
.XLSX files: An easy-to-read data format perfect for system testing or small batches of data.
Post-processing: Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. Summing Up While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution.


Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills.


Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. Having worked on multiple technical drawing processing projects, it was a matter of time till an engineering drawing automation project came our way. What’s so special about engineering drawings, you ask? Geometric dimensioning and tolerancing annotations (GD&T) is your answer. These pesky labels often present a challenge when processing and extracting data from engineering drawings due to their position on a page and overall structure. But don’t fret — I’m here to share just how we managed to process GD&T annotations on engineering drawings with AI. Let’s start from the beginning though. Processing Unstructured Documents All digital documents can be separated into 2 types: structured and unstructured: Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. Structured documents On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. unstructured documents As you might have guessed, technical drawings are a classic example of an unstructured document: despite adhering to a strict set of standards, each drawing is different from another as they lack rigid structure. Coupled with a mix of typed and handwritten text data, special symbols, complex spreadsheets, and various annotations, technical drawings present a real challenge for extracting data automatically. The complex nature of technical drawings makes them the perfect candidate for AI data extraction. In fact, using neural models to detect and extract various data from the drawings is the only way to automate their processing. Modern computer vision models and a smart approach to product development can yield a powerful tool for quick processing of any technical drawing. A Problem With Ready-Made Tools One quick Google search will show you at least a couple of solutions for processing engineering drawings. Nearly all of them offer wide functionality and promise quick and accurate processing of complex data. At first glance, this may sound very promising: paying for a monthly subscription to process engineering drawings with high precision. In practice, however, things are often not as smooth. Ready-made tools often struggle to detect and process rotated elements as their algorithms are only trained to process the “common denominator”, which, in our case, is an engineering drawing with labels and annotations positioned horizontally. Therefore using a ready-made solution is only suitable for those whose drawings are relatively simple and include only standard data. Any deviation from the “common denominator” will present a challenge for a ready-made tool. Feature Extraction From Engineering Drawings This exact situation happened to one of our clients: solutions for processing engineering drawings on the market don’t answer the needs of processing complex or non-standard drawings, leaving them with poor data recognition results. GD&T annotations carry a lot of very important information that is vital to extract from the drawing for further processing, but their position on the page, in our case, them being positioned at an angle, throws a wrench into the process of analyzing drawings by a premade AI tool. This is where custom AI development comes into play: AI models trained to detect and extract information from your specific document can solve (almost) any challenge a ready-made tool struggles with. Here’s how we’ve solved one of the challenges of processing engineering drawings with custom AI model development — extraction of GD&T annotations placed at an angle. Step 1: Detecting Annotation Position The first step is to locate the position of annotations on the drawing. AI models can be trained to detect the location of annotations irrespective of their position or rotation angle. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Step 2: Detect Rotation Angle Here’s the important part: detecting how the annotation is rotated. The AI model needs to calculate the rotation angle and rotate the annotation to make it horizontal. The cut-out PNGs are then passed on for further processing: Step 3: Extracting Data From Annotations After all annotations are detected, rotated, and extracted from the drawing, they are run through a symbol detection engine. Tesseract is a good choice for this as it provides high accuracy of recognition and can work with multiline text and symbols of different heights. First, you need to find the exact area where the text is located to improve the symbol recognition process. I would recommend using OpenCV as it handles these tasks very well and is relatively easy to work with. Next, the detected text area is handed over to the OCR engine to extract all text and symbols. Step 4: Analyzing Data An array of letters, numbers, and symbols need to be interpreted to provide “digestible” data that humans — or a data management system — can understand and process. Detected symbols are separated into groups forming part dimensions, tolerances, fits, and radii. Step 5: Data Management Data extracted by an AI system needs to be extracted according to your needs: JSON files: Perfect for importing the data into existing software, .XLSX files: An easy-to-read data format perfect for system testing or small batches of data. Post-processing: Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. JSON files : Perfect for importing the data into existing software, JSON files .XLSX files : An easy-to-read data format perfect for system testing or small batches of data. .XLSX files Post-processing : Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. Post-processing Summing Up While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here . how AI can help process architectural drawings here how AI can help process architectural drawings here

How To Process Engineering Drawings With AI

কিভাবে AI দিয়ে ইঞ্জিনিয়ারিং ড্রয়িং প্রসেস করবেন

So verarbeiten Sie technische Zeichnungen mit KI

Cómo procesar dibujos de ingeniería con IA

Comment traiter des dessins techniques avec l'IA

AI के साथ इंजीनियरिंग ड्राइंग को कैसे प्रोसेस करें

AI でエンジニアリング図面を処理する方法

AI로 엔지니어링 도면을 처리하는 방법

Como processar desenhos de engenharia com IA

Как обрабатывать инженерные чертежи с помощью ИИ

Yapay Zeka ile Mühendislik Çizimleri Nasıl İşlenir

Cách xử lý bản vẽ kỹ thuật bằng AI

如何利用人工智能处理工程图纸

In Papers, Please, a popular video game, you play the role of a customs officer checking documents according to increasingly complex rules. The main mechanic of the game is checking documents for compliance with all standards, such as the correct date and place of issue, compliance with the person’s first and last name on all documents, visa validity, whether the person is on “black” lists, and more. The game has attracted thousands of users around the world with its original style and unusual game mechanics, but for some people, this game would seem like a real nightmare because in real life, in their real workplace, they do the same thing. What Is KYC and Who Needs It KYC stands for Know Your Customer — a set of practices targeted towards obtaining as much information about a new customer as possible to weed out unreliable customers and reduce fraud. A comprehensive background check of a client, including their documents, bank history, and even activity on social networks, is included in the standard set of KYC practices aimed at reducing risks, which can be different for each business. Thus, a bank employee is forced to check not only the documents provided by the client but also interact with authorities in order to find out how respectable the person is and how likely they are to pay a loan back, and analyze their behavioral characteristics, for example, their behavior on social media. Online service managers, like online marketplaces and booking sites, on the other hand, do not need to analyze the user’s credit history, but it is important for them to know that their client does not use various loopholes, for example, does not register several times in a row using a temporary email to take advantage of a promo code, or does not register another account in order to leave negative feedback in order to lower the rating of the service. For some businesses, for example, car sharing and any Uber-like services, it is important to know who exactly uses their services (or provides them). For example, Uber needs to know who was driving the car when it got into an accident. Crypto exchanges, despite rejecting almost any regulations aimed at them, are forced to turn to KYC practices to identify clients due to local and global legal pressure. For example, to register on Binance, you not only need to enter your name and date of birth but also submit a photo of your ID, upload a selfie, and go through face verification using a webcam. Online dating sites and apps also strive to implement as many KYC practices as possible into the account registration process in order to avoid catfishing because this phenomenon can greatly reduce the number of users and undermine the reputation of the service. How It Usually Works KYC measures, in fact, are often very simple, both for the client and for the one who implements them. For example, verifying an email address via a message with a code or a phone number via a text message are the simplest (and most unreliable) examples of KYC. These measures are easily abused: everyone knows about temporary email services that are used to create multiple accounts in one service, which can be very useful for endlessly extending the free period of using the service, for example. These measures are not enough for businesses that value their user base. Let's talk about several options for making user verification more complex at the registration stage. Step 1: Are you human? The simplest step is the implementation of Captcha, which at least partially solves the problem with the mass registration of bots. This is very simple, cheap, and quite effective, with the reduction of bots noticeable almost instantly. However, Captcha alone cannot solve the problem of multi-accounts, a situation where one person registers in the system several times with malicious intent. Step 2: Determine IP Checking your clients' IPs is the next step in weeding out unreliable users. If different users regularly log into your service from the same IP, this may raise doubts about their integrity and purpose for using the service. In addition, detecting IPs of popular VPN services can also help reduce bots and unreliable customers. Step 3: Email and phone number verification Here, we are no longer talking about checking the user’s access to the specified phone number and email address but a more in-depth data verification. The validity of the email domain, the email registration date, and the connection with accounts on social networks are often checked. A phone number can also provide quite a lot of useful data, such as cellular operator, country, and connection to social networks. This way, you can identify those who created an email only for registration or use a burner phone. Step 4: Papers, please! This stage of improving verification significantly increases the number of steps during registration for the user, but it also increases the reliability of the service and significantly reduces the number of unscrupulous users within it. The person is asked to provide an identification document and enter their details when registering. Checking the validity of IDs is generally a simple task, as data on valid and invalid documents is available online. Step 5: Show your face There is still room for loopholes in the previous step. Nothing prevents a user who has been banned from a certain service but who ardently wants to continue using it from borrowing a passport from an unsuspecting grandmother and entering her information when registering a second account. This situation is extremely unpleasant for services seeking to maintain a healthy user base, and therefore, there needs to be an additional step: facial recognition. The most convenient way to implement facial recognition is in a mobile application. When registering, the user enters their ID information, takes a photo of the ID, and then the app asks them to make faces at the camera, distracting them from the facial recognition process, during which the following is performed: Comparison of the person’s face with their photo from the ID to determine whether the document belongs to them, Comparison with the database of faces of already registered users so that a person does not register several times using different documents, like a driver’s license instead of an ID, Check for liveness or whether a real person is in front of a camera and not their photo or a prerecorded video. Smart KYC: Real-life application and development Having had an opportunity to work on multiple AI-powered KYC systems for both web and mobile applications, I have a good understanding of how theory applies to practice. All of the steps outlined above are used in real-life applications, from simple email verification to facial recognition. While there are many ways to implement these practices in applications, I’ve compiled techniques and technologies behind smart KYC in modern systems. ID Recognition There are many different tools for document detection, all of which are geared toward different document types and different use cases. Most of them are paid, too, which is not ideal for a KYC system since the cost for each onboarding client would quickly pile up. One of the tools best suited for this application is Google Vision API, which detects text in any language and creates a bounding box around each word. Google Vision allows up to 1000 documents to be processed monthly for free, which should cover a decent chunk of new users for many services. To begin with, using the user-provided passport photo, we use OpenCV to determine the outline of the passport. This selection may not always go smoothly, for example, if the photo was taken against a colorful background or there are other objects nearby. In this case, we find the largest contour and believe that this is the passport. The resulting image is passed on to Google Vision, which identifies and highlights each word separately. After Google Vision detects each word, you can search for keywords like “Name”, “Surname”, “Date of Birth”, “Place Of Birth”, etc. Each passport type, which equals to country of origin, has its own config file with the following information: Keywords, or field titles, Spaces and line spacing Concerning spaces Do you think that countries differ from each other in culture, history, and their people? No, they differ in the size of spaces and line spacing in the passports of their citizens. For example, Romanian passports have short spaces, while Swedish passports have long spaces. The size of spaces is important to find the necessary information: after a keyword, in this case, field name, is found, the system looks for its value or field content, which can be either to the right of the keyword or below. Knowing the size of spaces and line spacing of a certain type of passport, we know how much to move to the right or down to find the value of a keyword. The size of spaces and intervals is not absolute but relative (depending on the size of the image). Facial recognition Another important aspect of smart KYC is facial recognition. With today’s abundance of excellent facial detection algorithms, there’s no need to reinvent the wheel and create your own bespoke solution, it’s much more cost-efficient to use a pretrained model for this job. For example, FaceNet is a good solution for the task of comparing two face photos and determining if it’s the same person or not. A simple script can extract a photo from an ID, while FaceNet (or any facial recognition model of your choice) analyzes the user's face using a smartphone camera and compares the two images to determine if the ID actually belongs to the user. Liveness detection Another important task when it comes to KYC is liveness detection, which is a set of techniques aimed at determining whether a source is a live human or a fake presentation. This is a vital step to eliminating multi-account fraud and preventing identity fraud. Liveness checks usually utilize motion detection and face detection algorithms to confirm the user’s identity. The easiest — and most effective — way to implement liveness detection is by prompting a user to perform a series of simple movements, like turning the head left or right, winking, smiling, etc. Again, there are several algorithms and models for movement detection on the market; it's a matter of choosing the right one for the task at hand. I recommend Google ML Kit as it's very easy to use and implement into an app. Summing Up Modern AI-based services and technologies make it possible to quickly and inexpensively implement a system that will eliminate account and identity fraud and improve KYC efforts. Well-implemented KYC measures are easy for the user to follow and complete and are effective enough to weed out users with bad intentions.

Papers, Please! - Know Your Customer With AI

Stories, News & Mentions about KOKOJI

Kokoji’s Stories on HackerNoon