Kokoji

kokoji

So, let’s start! Tell us a bit about yourself. For example, name, profession, and personal interests. My name is Oleg Kokorin; I’m a CEO of Businessware Technologies (a custom software development company) and a machine learning engineer. My main interests are in teaching computers to see and write, diving, and traveling the world! I also frequently participate in Kaggle competitions. Interesting! What was your latest Hackernoon Top story about? My latest article -- an animal recognition app we at Businessware Technologies have developed for a startup. In this article, I describe how we went from nothing to a powerful computer vision system capable of detecting animals in real-time. was about Birdsy Do you usually write on similar topics? If not, what do you usually write about? Yes, my blog here on HackerNoon is focused on the topics of AI, machine learning, and computer vision, specifically on how to create an AI product from scratch, as it’s something I’ve had a lot of experience with. I also enjoy writing guides and other helpful content for AI startups as I feel like I have amassed enough experience and knowledge worth sharing with upcoming entrepreneurs. Great! What is your usual writing routine like (if you have one?) I don’t really have a routine, but I always talk to my team from our and ML department as they are a bright bunch that can give a unique perspective as people who are every day deep in the trenches of AI development. This can either give me ideas for new articles or give me data for an article I’m already working on. AI For example, for my latest article, I have interviewed our top machine learning engineer who had hands-on experience with developing Birdsy. This made the article a lot more interesting, as personal experiences often are way more interesting than technical details. Being a writer in tech can be a challenge. It’s not often our main role, but an addition to another one. What is the biggest challenge you have when it comes to writing? Foor me, one of the biggest challenges when it comes to writing is making my articles engaging and interesting to read. We at Businessware Technologies have worked on so many interesting projects that have provided me with a lot of insights that I would love to share, but it’s often challenging to relay dry technical mumbo jumbo into an engaging text worth reading. I hope to get good enough with my writing that even the most tech-heavy text would be interesting to read for the average user. What is the next thing you hope to achieve in your career? I’m always looking forward to new projects with our clients, but at this moment, I’m excited to work more on predictive AI projects, i.e., artificial intelligence systems capable of predicting outcomes based on previous data. I’ve been focusing on computer vision systems for a good while; I think it’s about time to expand my horizons. Wow, that’s admirable. Now, something more casual: What is your guilty pleasure of choice? I enjoy a good show! Silicon Valley is at the top, for sure. Do you have a non-tech-related hobby? If yes, what is it? I enjoy diving and snorkeling! Recently-ish, I went to the Maldives and got to go snorkeling in the beautiful waters there. There’s nothing like swimming underwater with local marine life! What can the Hacker Noon community expect to read from you next? I’m planning to write about how we developed an AI for detecting technical drawings. Stay tuned! What’s your opinion on HackerNoon as a platform for writers? So far, it’s been great! I especially like the editor; it makes it very easy to format and edit an article. Thanks for taking time to join our “ ” series. It was a pleasure. Do you have any closing words? Meet the Writer Thank you for this opportunity!

Meet the Writer: HackerNoon's Contributor Oleg Kokorin, CEO of Businessware Technologies

The Greedy Miner’s Dilemma: Flaws in DAG-Based Consensus Protocols

The Hypersonic Drone Market: How the Axis of Innovation Is Forming Around It

This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.
Authors:
(1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. Table of Links Abstract and Introduction
Grassroots Protocols
Grassroots Implementation of Grassroots Dissemination
Applications of Grassroots Dissemination Supporting Digital Sovereignty
Future Work & Discussion
References
A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems
B. All-to-All Dissemination is not Grassroots
C. Proofs References 1 Martin R Albrecht, Jorge Blasco, Rikke Bjerg Jensen, and Lenka Mareková. Mesh messaging in large-scale protests: Breaking bridgefy. In Cryptographers’ Track at the RSA Conference, pages 375–398. Springer, 2021. 2 Juan Benet. Ipfs-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561, 2014. 3 Sonja Buchegger, Doris Schiöberg, Le-Hung Vu, and Anwitaman Datta. Peerson: P2p social networking: early experiences and insights. In Proceedings of the Second ACM EuroSys Workshop on Social Network Systems, pages 46–52, 2009. 4 Vitalik Buterin et al. A next-generation smart contract and decentralized application platform. white paper, 3(37), 2014. 5 Manuel Castells. The city and the grassroots: a cross-cultural theory of urban social movements. Number 7. Univ of California Press, 1983. 6 Miguel Castro, Barbara Liskov, et al. Practical byzantine fault tolerance. In OsDI, volume 99, pages 173–186, 1999. 7 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Constructing scalable overlays for pub-sub with many topics. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, pages 109–118, 2007. 8 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Spidercast: a scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the 2007 inaugural international conference on Distributed event-based systems, pages 14–25, 2007. 9 Bram Cohen. Incentives build robustness in bittorrent. In Workshop on Economics of Peer-to-Peer systems, volume 6, pages 68–72. Berkeley, CA, USA, 2003. 10 Alan Demers, Karin Petersen, Mike Spreitzer, Doug Terry, Marvin Theimer, and Brent Welch. The bayou architecture: Support for data sharing among mobile users. In 1994 First Workshop on Mobile Computing Systems and Applications, pages 2–7. IEEE, 1994. 11 Bitcoin Developer. P2p network, Retrieved 20222. URL: https://developer.bitcoin.org/ devguide/p2p_network.html. 12 Ethereum Developer. go-ethereum/params/bootnodes.go, Retrieved 2023. URL: https://github.com/ethereum/go-ethereum/blob/master/params/bootnodes.go. 13 Adem Efe Gencer, Soumya Basu, Ittay Eyal, Robbert van Renesse, and Emin Gün Sirer. Decentralization in bitcoin and ethereum networks. In International Conference on Financial Cryptography and Data Security, pages 439–457. Springer, 2018. 14 Gabriele Goldacker. Digitale souveränität [white paper of kompetenzzentrum öffentliche it (öfit)]. retrieved april 20, 2021, 2017. 15 Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. All you need is dag, 2021. arXiv:2102.08325. 16 Idit Keidar, Oded Naor, and Ehud Shapiro. Cordial miners: A family of simple and efficient consensus protocols for every eventuality. arXiv preprint arXiv:2205.09174, 2022. 17 Antony Lewis. A gentle introduction to self-sovereign identity. 2017. URL: https://bitsonblocks.net/2017/05/17/a-gentle-introduction-to-self-sovereign-identity/. 18 Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2019. 19 Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani. Epidemic processes in complex networks. Reviews of modern physics, 87(3):925, 2015. 20 Julia Pohle and Thorsten Thiel. Digital sovereignty. Internet Policy Review: Journal on Internet Regulation, 9(4):1–19, 2020. 21 Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. Challenges in the decentralised web: The mastodon case. In Proceedings of the Internet Measurement Conference, pages 217–229, 2019. 22 Vinay Setty, Maarten van Steen, Roman Vitenberg, and Spyros Voulgaris. Poldercast: Fast, robust, and scalable architecture for p2p topic-based pub/sub. In ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 271–291. Springer, 2012. 23 Devavrat Shah et al. Gossip algorithms. Foundations and Trends® in Networking, 3(1):1–125, 2009. 24 Ehud Shapiro. Multiagent transition systems: Protocol-stack mathematics for distributed computing. arXiv preprint arXiv:2112.13650, 2021. 25 Ehud Shapiro. Sovereign personal cryptocurrencies: A grassroots foundation for a digital economy. arXiv preprint arXiv:2202.05619, 2022. 26 Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. Conflict-free replicated data types. In Symposium on Self-Stabilizing Systems, pages 386–400. Springer, 2011. 27 Robert Shostak, Marshall Pease, and Leslie Lamport. The byzantine generals problem. ACM Transactions on Programming Languages and Systems, 4(3):382–401, 1982. 28 Ion Stoica, Robert Morris, David Karger, M Frans Kaashoek, and Hari Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM computer communication review, 31(4):149–160, 2001. 29 Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. Hotstuff: Bft consensus with linearity and responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, pages 347–356, 2019. 30 Shoshana Zuboff. The age of surveillance capitalism: The fight for a human future at the new frontier of power: Barack Obama’s books of 2019. Profile books, 2019. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: Authors: (1) Ehud Shapiro,  Department of Computer Science and Applied Math, Weizmann Institute of Science, Israel and ehud.shapiro@weizmann.ac.il. Table of Links Abstract and Introduction Grassroots Protocols Grassroots Implementation of Grassroots Dissemination Applications of Grassroots Dissemination Supporting Digital Sovereignty Future Work & Discussion References A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems B. All-to-All Dissemination is not Grassroots C. Proofs Abstract and Introduction Abstract and Introduction Grassroots Protocols Grassroots Protocols Grassroots Implementation of Grassroots Dissemination Grassroots Implementation of Grassroots Dissemination Applications of Grassroots Dissemination Supporting Digital Sovereignty Applications of Grassroots Dissemination Supporting Digital Sovereignty Future Work & Discussion Future Work & Discussion References References A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems A. Preliminaries: Asynchronous Distributed Multiagent Transition Systems B. All-to-All Dissemination is not Grassroots B. All-to-All Dissemination is not Grassroots C. Proofs C. Proofs References 1 Martin R Albrecht, Jorge Blasco, Rikke Bjerg Jensen, and Lenka Mareková. Mesh messaging in large-scale protests: Breaking bridgefy. In Cryptographers’ Track at the RSA Conference, pages 375–398. Springer, 2021. 2 Juan Benet. Ipfs-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561, 2014. 3 Sonja Buchegger, Doris Schiöberg, Le-Hung Vu, and Anwitaman Datta. Peerson: P2p social networking: early experiences and insights. In Proceedings of the Second ACM EuroSys Workshop on Social Network Systems, pages 46–52, 2009. 4 Vitalik Buterin et al. A next-generation smart contract and decentralized application platform. white paper, 3(37), 2014. 5 Manuel Castells. The city and the grassroots: a cross-cultural theory of urban social movements. Number 7. Univ of California Press, 1983. 6 Miguel Castro, Barbara Liskov, et al. Practical byzantine fault tolerance. In OsDI, volume 99, pages 173–186, 1999. 7 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Constructing scalable overlays for pub-sub with many topics. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, pages 109–118, 2007. 8 Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Spidercast: a scalable interest-aware overlay for topic-based pub/sub communication. In Proceedings of the 2007 inaugural international conference on Distributed event-based systems, pages 14–25, 2007. 9 Bram Cohen. Incentives build robustness in bittorrent. In Workshop on Economics of Peer-to-Peer systems, volume 6, pages 68–72. Berkeley, CA, USA, 2003. 10 Alan Demers, Karin Petersen, Mike Spreitzer, Doug Terry, Marvin Theimer, and Brent Welch. The bayou architecture: Support for data sharing among mobile users. In 1994 First Workshop on Mobile Computing Systems and Applications, pages 2–7. IEEE, 1994. 11 Bitcoin Developer. P2p network, Retrieved 20222. URL: https://developer.bitcoin.org/ devguide/p2p_network.html. 12 Ethereum Developer. go-ethereum/params/bootnodes.go, Retrieved 2023. URL: https://github.com/ethereum/go-ethereum/blob/master/params/bootnodes.go. 13 Adem Efe Gencer, Soumya Basu, Ittay Eyal, Robbert van Renesse, and Emin Gün Sirer. Decentralization in bitcoin and ethereum networks. In International Conference on Financial Cryptography and Data Security, pages 439–457. Springer, 2018. 14 Gabriele Goldacker. Digitale souveränität [white paper of kompetenzzentrum öffentliche it (öfit)]. retrieved april 20, 2021, 2017. 15 Idit Keidar, Eleftherios Kokoris-Kogias, Oded Naor, and Alexander Spiegelman. All you need is dag, 2021. arXiv:2102.08325. 16 Idit Keidar, Oded Naor, and Ehud Shapiro. Cordial miners: A family of simple and efficient consensus protocols for every eventuality. arXiv preprint arXiv:2205.09174, 2022. 17 Antony Lewis. A gentle introduction to self-sovereign identity. 2017. URL: https://bitsonblocks.net/2017/05/17/a-gentle-introduction-to-self-sovereign-identity/. 18 Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2019. 19 Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani. Epidemic processes in complex networks. Reviews of modern physics, 87(3):925, 2015. 20 Julia Pohle and Thorsten Thiel. Digital sovereignty. Internet Policy Review: Journal on Internet Regulation, 9(4):1–19, 2020. 21 Aravindh Raman, Sagar Joglekar, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. Challenges in the decentralised web: The mastodon case. In Proceedings of the Internet Measurement Conference, pages 217–229, 2019. 22 Vinay Setty, Maarten van Steen, Roman Vitenberg, and Spyros Voulgaris. Poldercast: Fast, robust, and scalable architecture for p2p topic-based pub/sub. In ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, pages 271–291. Springer, 2012. 23 Devavrat Shah et al. Gossip algorithms. Foundations and Trends® in Networking, 3(1):1–125, 2009. 24 Ehud Shapiro. Multiagent transition systems: Protocol-stack mathematics for distributed computing. arXiv preprint arXiv:2112.13650, 2021. 25 Ehud Shapiro. Sovereign personal cryptocurrencies: A grassroots foundation for a digital economy. arXiv preprint arXiv:2202.05619, 2022. 26 Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. Conflict-free replicated data types. In Symposium on Self-Stabilizing Systems, pages 386–400. Springer, 2011. 27 Robert Shostak, Marshall Pease, and Leslie Lamport. The byzantine generals problem. ACM Transactions on Programming Languages and Systems, 4(3):382–401, 1982. 28 Ion Stoica, Robert Morris, David Karger, M Frans Kaashoek, and Hari Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM computer communication review, 31(4):149–160, 2001. 29 Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. Hotstuff: Bft consensus with linearity and responsiveness. In Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, pages 347–356, 2019. 30 Shoshana Zuboff. The age of surveillance capitalism: The fight for a human future at the new frontier of power: Barack Obama’s books of 2019. Profile books, 2019.

Grassroots Distributed Systems for Digital Sovereignty: References

PIECHAIN Demo: Exploring a Practical Blockchain Interoperability Framework

পাইচেন ডেমো: একটি ব্যবহারিক ব্লকচেইন ইন্টারঅপারেবিলিটি ফ্রেমওয়ার্ক অন্বেষণ করা

PIECHAIN-Demo: Erkundung eines praktischen Blockchain-Interoperabilitätsrahmens

Demostración de PIECHAIN: Explorando un marco práctico de interoperabilidad de Blockchain

Démo PIECHAIN : Explorer un cadre pratique d'interopérabilité de la blockchain

PIECHAIN डेमो: एक व्यावहारिक ब्लॉकचेन इंटरऑपरेबिलिटी फ्रेमवर्क की खोज

PIECHAIN デモ: 実用的なブロックチェーン相互運用性フレームワークの探究

PIECHAIN 데모: 실용적인 블록체인 상호 운용성 프레임워크 탐색

Demonstração PIECHAIN: Explorando uma estrutura prática de interoperabilidade Blockchain

Демонстрация PIECHAIN: изучение практической структуры взаимодействия блокчейнов

PIECHAIN Demosu: Pratik Bir Blockchain Birlikte Çalışabilirlik Çerçevesini Keşfetmek

Bản trình diễn PIECHAIN: Khám phá Khung tương tác Blockchain thực tế

PIECHAIN 演示：探索实用的区块链互操作性框架

**PALO ALTO, California, March 20th, 2024/Chainwire/--**Pilotfish, a prototype Sui extension, was able to grow smart contract execution throughput in direct proportion to the number of machines used. Mysten Labs, a team of leading distributed systems, programming languages, and cryptography experts and originators of the Sui Network, today announced a landmark achievement in scaling blockchain capacity. During testing and development in a Sui blockchain environment, Pilotfish, a prototype Sui extension, increased throughput by 8x when backed by 8 machines, successfully illustrating the possibility of linear scaling. Notably, the latency per transaction decreased as more machines were added during the test, proving the viability of linear horizontal scaling for low latency blockchain transactions for the first time ever on any blockchain. This proof-of-concept, was formally presented as an academic preprint titled, " ," and initially shared on January 29, 2024 as an article on the open access archive arXiv. Pilotfish: Distributed Transaction Execution for Lazy Blockchains “Pilotfish is a game changer as the blockchain industry seeks to achieve the same dynamic capacity increase currently possible only with centralized server technologies,” said Lefteris Kokoris-Kogias, one of the authors of the paper and researcher at Mysten Labs. "Pilotfish has the potential to push throughput well beyond what is currently at the forefront of high-performance blockchains, enabling transaction throughput levels no other blockchain existing today can achieve." Sui leads the industry in throughput by leveraging multiple cores for parallel transaction execution within individual validators. Using this approach, within months following mainnet, Sui reached 65.8 million transaction executions in a single day—the most of any blockchain ever. However, a single machine can only incorporate a finite number of cores, so throughput gains via this strategy are limited. Moreover, the machines capable of accommodating large numbers of cores are expensive to buy and use. The breakthrough solution that Pilotfish offers enables a single validator to employ multiple servers simultaneously. This new technology not only makes elastic capacity possible, but its auto-scaling architecture, akin to centralized solutions like AWS or Heroku, allows the use of smaller, cheaper servers available on demand. Nearly all other blockchains aiming to scale use some form of batching—grouping transactions together to be added all at once. The downside of this method is significant additional latency. Instead of transactions moving directly to finality, they are delayed waiting for others to be batched for execution with them. Unlike these other scaling solutions, Pilotfish achieves potentially linear throughput scaling without increasing latency. In the simplest terms, with the horizontal scaling enabled by Pilotfish, simply by engaging the requisite number of machines, a validator can adapt its throughput and resource use to the use case at hand. Alberto Sonnino, a research scientist at Mysten Labs, said, "Until now the only option to increase a validator’s capacity has been to upgrade to a more powerful machine. Pilotfish removes that constraint, making possible horizontal scaling by simply adding servers and without delaying execution. With Pilotfish, the question is no longer whether a level of throughput is possible, only how many servers it will require." Pilotfish achieves its remarkable scalability by enabling a single validator to execute transactions across multiple machines. Specifically, Pilotfish partitions the validation function into three key roles: (1) the Primary, which handles sequencing transactions using a high-throughput consensus, (2) SequencingWorkers (SWs), which store transactions and dispatch them for execution; and (3) ExecutionWorkers (EWs), which store the blockchain state and execute transactions received from the SWs. Different components may reside on multiple different machines to increase the capacity of the system with more precision. In experimental results, Pilotfish has already achieved remarkable results in terms of linear scaling, low latency, and support for various workloads. In testing, Pilotfish reduced network response times and maintained a latency envelope below 20ms. Future plans include enhancements like multiple SequencingWorkers, shard replication, and ultra-fast networking. Contact Mysten Labs lexi.wangler@mystenlabs.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here.

Mysten Labs Technology Prototype On Sui Provides First Proof Of Elastic Blockchain Scaling

Innovating Identity Verification: An Introduction to Proof of Personhood (PoP) Protocols

3 Approaches To Digitizing Newspapers & Magazines

How To Find The Best LLM For Your Project

Having worked on multiple technical drawing processing projects, it was a matter of time till an engineering drawing automation project came our way. What’s so special about engineering drawings, you ask? Geometric dimensioning and tolerancing annotations (GD&T) is your answer. These pesky labels often present a challenge when processing and extracting data from engineering drawings due to their position on a page and overall structure. But don’t fret — I’m here to share just how we managed to process GD&T annotations on engineering drawings with AI. Let’s start from the beginning though. Processing Unstructured Documents All digital documents can be separated into 2 types: structured and unstructured: Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. As you might have guessed, technical drawings are a classic example of an unstructured document: despite adhering to a strict set of standards, each drawing is different from another as they lack rigid structure. Coupled with a mix of typed and handwritten text data, special symbols, complex spreadsheets, and various annotations, technical drawings present a real challenge for extracting data automatically. The complex nature of technical drawings makes them the perfect candidate for AI data extraction. In fact, using neural models to detect and extract various data from the drawings is the only way to automate their processing. Modern computer vision models and a smart approach to product development can yield a powerful tool for quick processing of any technical drawing. A Problem With Ready-Made Tools One quick Google search will show you at least a couple of solutions for processing engineering drawings. Nearly all of them offer wide functionality and promise quick and accurate processing of complex data. At first glance, this may sound very promising: paying for a monthly subscription to process engineering drawings with high precision. In practice, however, things are often not as smooth. Ready-made tools often struggle to detect and process rotated elements as their algorithms are only trained to process the “common denominator”, which, in our case, is an engineering drawing with labels and annotations positioned horizontally. Therefore using a ready-made solution is only suitable for those whose drawings are relatively simple and include only standard data. Any deviation from the “common denominator” will present a challenge for a ready-made tool. Feature Extraction From Engineering Drawings This exact situation happened to one of our clients: solutions for processing engineering drawings on the market don’t answer the needs of processing complex or non-standard drawings, leaving them with poor data recognition results. GD&T annotations carry a lot of very important information that is vital to extract from the drawing for further processing, but their position on the page, in our case, them being positioned at an angle, throws a wrench into the process of analyzing drawings by a premade AI tool. This is where custom AI development comes into play: AI models trained to detect and extract information from your specific document can solve (almost) any challenge a ready-made tool struggles with. Here’s how we’ve solved one of the challenges of processing engineering drawings with custom AI model development — extraction of GD&T annotations placed at an angle. Step 1: Detecting Annotation Position The first step is to locate the position of annotations on the drawing. AI models can be trained to detect the location of annotations irrespective of their position or rotation angle. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Step 2: Detect Rotation Angle Here’s the important part: detecting how the annotation is rotated. The AI model needs to calculate the rotation angle and rotate the annotation to make it horizontal. The cut-out PNGs are then passed on for further processing: Step 3: Extracting Data From Annotations After all annotations are detected, rotated, and extracted from the drawing, they are run through a symbol detection engine. Tesseract is a good choice for this as it provides high accuracy of recognition and can work with multiline text and symbols of different heights. First, you need to find the exact area where the text is located to improve the symbol recognition process. I would recommend using OpenCV as it handles these tasks very well and is relatively easy to work with. Next, the detected text area is handed over to the OCR engine to extract all text and symbols. Step 4: Analyzing Data An array of letters, numbers, and symbols need to be interpreted to provide “digestible” data that humans — or a data management system — can understand and process. Detected symbols are separated into groups forming part dimensions, tolerances, fits, and radii. Step 5: Data Management Data extracted by an AI system needs to be extracted according to your needs: JSON files: Perfect for importing the data into existing software,
.XLSX files: An easy-to-read data format perfect for system testing or small batches of data.
Post-processing: Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. Summing Up While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution.


Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills.


Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. Having worked on multiple technical drawing processing projects, it was a matter of time till an engineering drawing automation project came our way. What’s so special about engineering drawings, you ask? Geometric dimensioning and tolerancing annotations (GD&T) is your answer. These pesky labels often present a challenge when processing and extracting data from engineering drawings due to their position on a page and overall structure. But don’t fret — I’m here to share just how we managed to process GD&T annotations on engineering drawings with AI. Let’s start from the beginning though. Processing Unstructured Documents All digital documents can be separated into 2 types: structured and unstructured: Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. Structured documents follow a predefined structure, making them easy to process and analyze with AI. Documents like forms, invoices, receipts, surveys, and contracts are all examples of structured documents. Structured documents On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. On the contrary, unstructured documents lack a consistent organization, making them inherently challenging to process automatically. Examples of unstructured documents include newspapers, research papers, and business reports. unstructured documents As you might have guessed, technical drawings are a classic example of an unstructured document: despite adhering to a strict set of standards, each drawing is different from another as they lack rigid structure. Coupled with a mix of typed and handwritten text data, special symbols, complex spreadsheets, and various annotations, technical drawings present a real challenge for extracting data automatically. The complex nature of technical drawings makes them the perfect candidate for AI data extraction. In fact, using neural models to detect and extract various data from the drawings is the only way to automate their processing. Modern computer vision models and a smart approach to product development can yield a powerful tool for quick processing of any technical drawing. A Problem With Ready-Made Tools One quick Google search will show you at least a couple of solutions for processing engineering drawings. Nearly all of them offer wide functionality and promise quick and accurate processing of complex data. At first glance, this may sound very promising: paying for a monthly subscription to process engineering drawings with high precision. In practice, however, things are often not as smooth. Ready-made tools often struggle to detect and process rotated elements as their algorithms are only trained to process the “common denominator”, which, in our case, is an engineering drawing with labels and annotations positioned horizontally. Therefore using a ready-made solution is only suitable for those whose drawings are relatively simple and include only standard data. Any deviation from the “common denominator” will present a challenge for a ready-made tool. Feature Extraction From Engineering Drawings This exact situation happened to one of our clients: solutions for processing engineering drawings on the market don’t answer the needs of processing complex or non-standard drawings, leaving them with poor data recognition results. GD&T annotations carry a lot of very important information that is vital to extract from the drawing for further processing, but their position on the page, in our case, them being positioned at an angle, throws a wrench into the process of analyzing drawings by a premade AI tool. This is where custom AI development comes into play: AI models trained to detect and extract information from your specific document can solve (almost) any challenge a ready-made tool struggles with. Here’s how we’ve solved one of the challenges of processing engineering drawings with custom AI model development — extraction of GD&T annotations placed at an angle. Step 1: Detecting Annotation Position The first step is to locate the position of annotations on the drawing. AI models can be trained to detect the location of annotations irrespective of their position or rotation angle. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Note: Multipage documents require an additional step of dividing the document into pages and differentiating between different engineering drawings. The same goes for documents that include multiple drawings on each page: you need to first run a model to detect each drawing and extract them from the document. Step 2: Detect Rotation Angle Here’s the important part: detecting how the annotation is rotated. The AI model needs to calculate the rotation angle and rotate the annotation to make it horizontal. The cut-out PNGs are then passed on for further processing: Step 3: Extracting Data From Annotations After all annotations are detected, rotated, and extracted from the drawing, they are run through a symbol detection engine. Tesseract is a good choice for this as it provides high accuracy of recognition and can work with multiline text and symbols of different heights. First, you need to find the exact area where the text is located to improve the symbol recognition process. I would recommend using OpenCV as it handles these tasks very well and is relatively easy to work with. Next, the detected text area is handed over to the OCR engine to extract all text and symbols. Step 4: Analyzing Data An array of letters, numbers, and symbols need to be interpreted to provide “digestible” data that humans — or a data management system — can understand and process. Detected symbols are separated into groups forming part dimensions, tolerances, fits, and radii. Step 5: Data Management Data extracted by an AI system needs to be extracted according to your needs: JSON files: Perfect for importing the data into existing software, .XLSX files: An easy-to-read data format perfect for system testing or small batches of data. Post-processing: Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. JSON files : Perfect for importing the data into existing software, JSON files .XLSX files : An easy-to-read data format perfect for system testing or small batches of data. .XLSX files Post-processing : Data is additionally processed to send it straight to a digital document handling system; great for those looking for a complete solution. Post-processing Summing Up While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. While the market is full of AI tools for processing documents, they only handle simple files well. Any deviation from the “norm” is better processed with a custom solution. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Custom AI models can handle virtually all data extraction tasks — given the right approach and developer skills. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here. Engineering drawings are not the only technical drawings I’ve written about, check out how AI can help process architectural drawings here . how AI can help process architectural drawings here how AI can help process architectural drawings here

How To Process Engineering Drawings With AI

কিভাবে AI দিয়ে ইঞ্জিনিয়ারিং ড্রয়িং প্রসেস করবেন

So verarbeiten Sie technische Zeichnungen mit KI

Cómo procesar dibujos de ingeniería con IA

Comment traiter des dessins techniques avec l'IA

AI के साथ इंजीनियरिंग ड्राइंग को कैसे प्रोसेस करें

AI でエンジニアリング図面を処理する方法

AI로 엔지니어링 도면을 처리하는 방법

Como processar desenhos de engenharia com IA

Как обрабатывать инженерные чертежи с помощью ИИ

Yapay Zeka ile Mühendislik Çizimleri Nasıl İşlenir

Cách xử lý bản vẽ kỹ thuật bằng AI

如何利用人工智能处理工程图纸

Stories, News & Mentions about KOKOJI

Kokoji’s Stories on HackerNoon