People’s Leasing Company PLC (PLC) is not only the market leader in Sri Lanka’s non-banking financial services (NBFS) space but is also at the forefront of the digital transformation in the sector. Further solidifying this reputation,

PLC Emerges as Best Digital Process Enabler at NQBSA Out-competing Banks and Others

Machine learning operations startup Domino Data Lab Inc. today announced a major update to its artificial intelligence development platform that it says will make it easier for companies to build generative AI models and other applications.

Domino Data bids to fast-track generative AI model development with project templates

DevOps Engineer roles and responsibilities encompass a wide range of tasks related to software development, deployment, and operations. We may be compensated by vendors who appear on this page through methods such as affiliate links or sponsored partnerships.

DevOps Engineer Roles and Responsibilities

As young as a newborn baby, Rwandans will soon acquire digital identity (ID) cards, replacing the current physical ID cards, as Rwanda continues to delve into technological solutions. Previously, officials said the development was only limited to children of five years and above.

Digital ID to be issued to newborn babies

Introduction Software development stands as the backbone of innovation. In today’s fast-paced world, where digital platforms, mobile applications, artificial intelligence, and the internet of things are advancing at an unprecedented pace,

Exploring the Future of Software Development: Pursuing a BS in Computer Software Technology

Bosses need to learn to stop messing with their software developers. It's a lesson that a few higher-ups learned the hard way after they decided to lay off one of their software devs just days before he was ready to release a big project he'd been working on.

Laid Off Software Developer Takes Code With Him, and Takes Down Company

Through this agreement, the two companies will collaborate in the industrial robotics field, jointly promoting the development of industrial robotic systems, the exploration of automation processes in industrial settings,

Neuromeka and KEBA Sign an MOU for the Joint Development of Industrial Robots

With the development of technology, it is now possible to design complicated apps without spending a lot of money, waiting for weeks or months, or hiring several engineers. Given the development of low-code and no-code platforms,

AI and No-Code: A Dynamic Duo for Digital Transformation

“Enterprise IT spending is recession-proof,” said John-David Lovelock, a VP Analyst at Gartner. He continues, “CEOs and CFOs, rather than cutting IT budgets, are increasing spending on digital business initiatives.” Despite economic fears, Gartner predicts spending on software will increase 11% in 2023, up to $879 billion from $790 billion in 2022.

Four Crucial Steps For Successful Software Implementation

In WiMi's multi-modal EEG-based hybrid BCI system with visual servo module, the generation of control commands is the core part of the whole system. Control command generation involves parsing the recognized EEG signals and mapping them to the corresponding robot actions.

WiMi Developed Multi-modal EEG-Based Hybrid BCI System with Visual Servo Module

MobiDev

mobidev

(*Kharkiv Regional State Administration after Russian Bombing*) Normally we publish articles by MobiDev’s experts about tech innovations and business use cases in software development, but this story will be about people, who found themselves in the middle of a war. On February 24th, Thursday, MobiDev employees woke up at 5 AM — Ukraine was attacked by Russia. Since that moment everyone has their own story to tell. Full stories of MobiDev employees can be found . here “With shaking hands we started to call our relatives, lying on the bathroom floor because it was the safest place during the bombing. During the next 24 hours my family and I stayed in the metro station, using it as a bomb shelter. We were sleeping on the freezing floor with our sleeping pads and blankets. After the sleepless nights when we came back home, it was clear that we had to run away, ” — remembers , BA Team Leader at MobiDev. Iryna Gurskaya It was not that everyone started to evacuate immediately. It took about a week for and his family before they decided to relocate. Danil Yaylo “On the 7th day of the war they started dropping banned bombs on the city. The road out of the city was very slow because of a traffic jam, but after 4 hours we came to a safe place, ” — that was the turning point for Danil when his family decided to move. For , Marketing Team Lead at MobiDev, that morning started with unexplainable anxiety: Julia Ramyalg “I woke up my kids trying to speak as calmly as possible. I called my mum asking her to come to my home. For the next few hours we were sitting in the hall between two concrete walls in my apartment, and waiting for something. ” Julia later remembers that having spent a few hours at the metro as a shelter, there was only one right decision — to take her family, and her dog before fleeing Kharkiv. The first two weeks were the hardest for the work organization as the general situation was quite unpredictable, but most employees were relocated to safety and even in the first days, . 65% kept on working MobiDev people acted to battle the enemy — some of our guys joined the troops, others were donating to the Army, and a lot of voluntary activities were launched within the company’s community. In their turn, clients almost immediately addressed AMs asking for ways to help and support Ukraine with real action. In parallel, as a part of a business contingency plan, MobiDev was organizing from Kharkiv. The whole crew of IT professionals has provided a safe journey for more than 70 people from Kharkiv to Chernivtsi, Ukraine. evacuation buses Sadly, this attack is not the first attempt by Russia to invade Ukraine. Back in 2014, Russia attacked Ukraine and occupied part of our territory (the Crimean Peninsula and parts of the Donetsk and Luhansk regions). At that time it was really unexpected, but Ukraine managed to stop the invasion and localized the conflict to those near-border regions. While Russia wages a war against Ukraine, most countries, businesses, and ordinary people stand with Ukraine. , Marketing Agent at MobiDev, is grateful for a huge response: Nana Hrytsenko “We are extremely lucky to have friends all over the world, who immediately started to ask what’s going on and how they can help. Actually, I’m safe in Poland at the moment — my friends provided me with everything I need.” Today most MobiDev employees have moved to the company’s offices in Chernivtsi and Lodz (Ukraine and Poland respectively). Some went to other safe locations across Ukraine and Europe. And they are already back at work. It’s just a matter of time until Ukraine wins this war, but at the moment what adds up to the success of the Ukrainian battle is the help of other countries. Like many other businesses, MobiDev warriors support the economic defense of the country, supplying the Ukrainian Army and volunteers with much-needed items. This is possible due to the continual and stable cooperation with our clients. This attitude really uplifted MobiDev teams, because, for us, such moves are a sign of trust and solidarity. The Ukrainian nation is united as never before and the support of other countries means the world to us!

MobiDev Employees Recount Their Stories about the War in Ukraine

Los empleados de MobiDev cuentan sus historias sobre la guerra en Ucrania

Les employés de MobiDev racontent leurs histoires sur la guerre en Ukraine

MobiDev के कर्मचारी यूक्रेन में युद्ध के बारे में अपनी कहानियां सुनाते हैं

MobiDev の従業員がウクライナでの戦争について語る

Funcionários da MobiDev contam suas histórias sobre a guerra na Ucrânia

Nhân viên MobiDev kể lại câu chuyện của họ về cuộc chiến ở Ukraine

MobiDev 员工讲述他们关于乌克兰战争的故事

As a startup founder in the sports industry, you may have heard about the potential of AI in fitness software over the past decade. However, incorporating into your fitness applications can still be confusing, especially if you haven’t worked with AI before. Today, we'd like to summarize what AI technology can offer to fitness applications and illustrate its use with some real-world examples. AI Virtual Coaching Applications AI coaching apps analyze exercise recordings in real-time, using computer vision and motion analysis. Users record their exercises with smartphones or wearables, capturing video or motion data. AI algorithms extract key points and track movements to assess form and biomechanics. By comparing to ideal standards, based on expert input or datasets, the app provides personalized, real-time feedback. This benefits users without access to in-person trainers, improving exercise technique and guidance. , a sports technology startup, is an example of an application that offers advanced specialized training for athletes. One of their innovative features is "comparative training," which allows athletes to compare their body mechanics and positioning with a proxy for perfect training. By using this app, athletes can learn and enhance their technique for better performance. BeONE Sports https://youtu.be/NbOgvXVYlH0?si=BAC4qL3UoC3ApsCH&embedable=true Personalized Workout Recommendations AI-powered personalized workout recommendation apps can generate customized workout plans. Users provide information about their fitness goals, exercise history, preferences, and limitations. This data forms a user profile that guides the AI algorithm. The algorithm then analyzes user data and identifies correlations between exercises, workout durations, intensity levels, and outcomes. This personalized approach increases user engagement, motivation, and the likelihood of achieving desired fitness outcomes. You can find many examples of AI-based personal training apps on the market like , Fitbod, Freeletics, Zing Coach, and more. Evolve AI Rehabilitation Solutions AI can be utilized in the rehabilitation and recovery process for individuals with injuries, disabilities, or those undergoing medical treatments. These apps provide personalized guidance, monitoring, and support to help users enhance their physical function and regain mobility. They often incorporate wearable devices or sensors to track movements and relevant metrics during exercises. AI algorithms analyze the collected data to monitor progress and offer performance feedback in real-time. It's still crucial to remember that these apps are intended to complement and enhance the rehabilitation process under the supervision of healthcare professionals rather than replacing professional medical advice and supervision. Our team at MobiDev has been developing an application for the rehabilitation and remote care of the elderly. This app incorporates a to detect and analyze pose movements. By comparing the user's exercise performance with a "virtual trainer," the app offers personalized recommendations on technique and injury prevention. human pose estimation model All-In-One Coaching Bots Coaching bots are examples of solutions that integrate various AI technologies, including human pose estimation and conversational AI. They can assist athletes in refining their exercise technique, correcting posture, selecting appropriate exercises, offering diet recommendations, and more. These comprehensive coaching bots leverage AI to provide the latest advancements in fitness technology, delivering a convenient, all-in-one app experience. Challenges and Future of Adopting AI in Fitness Despite all the exciting things that AI can bring to , the incorporation of this technology has certain challenges that you should be prepared for. building software products AI apps heavily rely on user-provided data, which can be challenging to input accurately. Motion analytics, for example, require users to follow strict filming rules to capture usable video. Poor lighting, inappropriate angles, and occlusive objects can result in frustrating retakes and inadequate input for the AI models. Technical Limitations: Fitness AI apps require a vast amount of data to recognize various body dimensions, sleeping patterns, and training capabilities. Collecting this data is an ongoing process, limiting the full potential of AI-enabled fitness apps until sufficient data is available. Data Accessibility: The lack of transparency in AI decision-making raises concerns about trusting AI coaching recommendations. Users may question the ethics and reasoning behind AI's suggestions. Developers face the challenge of making applications more trustworthy by providing explanations behind AI decisions. Trustworthiness of AI: : AI-enabled fitness apps that collect and store information about diet programming or rehabilitation procedures must navigate healthcare regulations and privacy concerns. The broader the scope of the AI fitness project, the more privacy considerations it encompasses. Privacy and Regulations At the same time, the rapid advancement of artificial intelligence presents significant opportunities for founders of fitness and sports startups. To capitalize on this trend, it’s a good idea to begin integrating smart functionality into your offerings now. By doing so, you can establish a strong foundation for growth in this emerging and progressive market.

How AI Transforms the Fitness Industry

কীভাবে এআই ফিটনেস শিল্পকে রূপান্তরিত করে

Wie KI die Fitnessbranche verändert

Cómo la IA transforma la industria del fitness

Comment l'IA transforme l'industrie du fitness

एआई फिटनेस उद्योग को कैसे बदल देता है

AI がフィットネス業界をどのように変革するか

AI가 피트니스 산업을 변화시키는 방법

Como a IA transforma a indústria do fitness

Как искусственный интеллект меняет фитнес-индустрию

Yapay Zeka Fitness Sektörünü Nasıl Dönüştürüyor?

AI biến đổi ngành thể hình như thế nào

人工智能如何改变健身行业

Scaling is the logical milestone for each software product. When the monolithic architecture prevents you from enjoying further growth, it’s time to consider migrating to microservices. Although microservices have significant advantages, their implementation may be tricky because of the complexity and absence of the silver bullet migration strategy. In this article, I’ll share my solution architect’s expertise in tackling the challenges of migrating from a monolith to microservices, as well as explain how to implement the project while preserving its security and reliability. From a Monolithic Architecture to Microservices: Migration Journey Let’s get straight to what can make the process of moving from a monolith to microservices as seamless as possible and start with the main five steps this transformation involves. STEP 1. CONDUCT TECH AND BUSINESS ANALYSIS Ensuring a successful migration to microservices begins by justifying the shift from a business perspective. Since every project possesses unique technical capabilities and limitations, it’s vital to consider that the move impacts the entire product architecture, which needs to be effective for future growth. It’s thus advisable to collaborate with both a business analyst and a technical expert to accurately evaluate current system requirements and draft an efficient development roadmap. While microservices have been in the limelight for years, the truth is they aren’t revolutionary for all projects. Few achieve the microservices ideal without some trade-offs. Although they’ve become somewhat of an industry buzzword, no one has established a perfect microservices environment. Yet, this shouldn’t deter businesses from aiming to transition if it aligns with their goals. It’s essential to manage expectations appropriately and assess if the envisioned tech implementation can meet the predetermined objectives. Moving to microservices is a sought-after . Starting a product directly with microservices might not be the optimal approach. Instead, one should consider beginning with a monolith or placing the core business logic within it. From here, extracting services becomes more manageable. Efforts to achieve perfectly isolated microservices might be overambitious, potentially leading to unnecessary complications. app modernization strategy For projects involving large teams, microservices can be the right choice. They provide scalability, not just in system architecture but also in team dynamics. A notable advantage of this approach is the ability to incorporate different technologies. Before moving to microservices, a comprehensive technical audit is imperative. This audit should pinpoint the current technology underpinning the product. It’s crucial to discern if this technology might pose limitations for future endeavors or if an alternative should be adopted for the microservices. Engaging with experts familiar with the particular technology can provide invaluable insights, ensuring a smooth and efficient architectural transformation. STEP 2. IDENTIFY PARTS OF THE SYSTEM TO BE TRANSFERRED TO MICROSERVICES The next step is to understand what parts of the system can be transitioned to microservices and plan how these microservices will interact (dependencies will describe how changes in one module affect other modules). Essentially, it’s about designing the product’s overarching architecture and prioritizing which services to extract. Microservices are most effective when they handle specific functionalities. Examples include real-time communication, data processing pipelines, background job processing, or interactions with external services. These functionalities might need data access from the monolith but are technologically distinct. STEP 3. BREAK THE MONOLITH TO MICROSERVICES Breaking a monolith into microservices can be done by applying two approaches. In the first one, we start by separating the desired functionality within the monolith and gradually disconnect it from other parts but keep it within the monolith. After disconnecting all ties and designing a new API, we move it out to launch as a separate microservice. This approach requires significant changes in the monolith during the process. The second approach includes making an independent copy of the desired functionality and developing it into a microservice while the original still runs in the monolith. Once the new microservice is fully functional and tested, we remove the original from the monolith. STEP 4. HANDLE DATA MANAGEMENT A key principle of microservices architecture is that each microservice should have its own dedicated database. Yet, breaking up a monolithic database can be challenging due to potential overlaps and interdependencies between database objects. Different microservices can use distinct databases, programming languages, and data storage solutions. Working with certain databases may be more complex than others, making it impractical to consolidate all data into a single database. Hence, specialized storage systems are often used for specific data types. Data management operations for microservices can be grouped according to the involved data-related patterns: DATABASE-PER-SERVICE The Database-per-Service pattern is a pivotal approach in microservice architecture, underscoring the importance of autonomy and encapsulation. By assigning a dedicated database to each microservice, it ensures data consistency and isolation and reduces contention among services. However, data integration and cross-service querying can become complex, requiring efficient communication protocols and well-defined interfaces. Additionally, database schema evolutions must be handled with care to avoid unintended service disruptions. But, with the right strategies in place, the Database-per-Service pattern can profoundly enhance scalability and fault tolerance in microservice ecosystems. SAGA The Saga pattern is a critical strategy in microservice transactions, addressing the challenges of maintaining data consistency across distributed systems. Instead of relying on traditional database transactions, it breaks operations into a series of isolated, compensatable actions. If a step fails, compensating transactions are triggered to ensure system consistency. While this decentralized approach bolsters system resilience and scalability, it demands careful orchestration and error handling to effectively manage potential failures. API COMPOSITION The API Composition pattern is a foundational method within microservices architecture that manages the challenge of data retrieval from multiple services. In an environment where each microservice is responsible for its unique piece of data, direct client-side data querying can be both complex and inefficient. The API Composition pattern addresses this by using an intermediary – often called an API composer or aggregator – to assemble the required data from various microservices into a unified response. By doing so, it provides clients with a singular access point, simplifying queries and streamlining data delivery. While this approach enhances the client experience and reduces direct microservice interactions, developers must ensure the composer remains efficient and doesn’t become a bottleneck or a single point of failure. CQRS Given that microservices advocate for decentralized data management, complexities can arise, especially when one service needs to both update data and query it.\ CQRS addresses this by segregating the responsibility of command operations (writes) from query operations (reads). In a microservices environment, this means a service can be optimized for its most common task: some services might be read-heavy, while others deal primarily with write operations. As a result, each can scale independently based on its workload. CQRS in microservices offers many advantages but introduces additional complexity, particularly in ensuring data consistency across services. Thus, its adoption should be thoughtfully considered within the context of the system’s specific requirements. EVENT SOURCING Event Sourcing emphasizes the capture and storage of all changes to the application state as a sequence of events. Instead of storing the current state of data in a domain, it stores a series of state transitions, enabling a system to reconstruct its state by playing back the events. In the context of microservices, this allows each service to maintain its own history, promoting autonomy and decoupling between services. As events become the single source of truth, they can be leveraged for multiple purposes, from analytics to audit trails. Additionally, this approach simplifies error handling and recovery mechanisms, as one can revert to a previous state or reprocess events. While powerful, the pattern necessitates careful consideration of event versioning and storage scalability. SHARED DATABASE The Shared Database anti-pattern arises when multiple microservices or components in an architecture interact directly with a common database instead of through APIs or messaging. This direct access not only compromises the autonomy of services but also introduces potential data integrity concerns. A system built on this approach may face security challenges, as services can inadvertently expose sensitive data. Such a design also complicates system evolution, turning seemingly minor database changes into complex tasks that demand coordination across multiple services. NEXT STEPS Whatever data management strategy is chosen, you need to avoid a Distributed Monolith. If services cannot be fully isolated, it often leads to more issues. Such a solution remains monolithic in terms of structure and databases. Even though it seems we have separated them, they end up having many connections, and most of the advantages of microservices are lost. Next, we need to through which the microservice will communicate with the monolith and other microservices. The referenced service provides data via an API that the requesting service requires. develop API interfaces STEP 4. ENSURE INTERSERVICE COMMUNICATION When designing interservices communication, consider the interaction process. Service interactions can be classified into two groups: One service processes each client request (one‑to‑one interaction) Each request is processed by a number of services (one‑to‑many interactions) Also, take into account the synchronous or asynchronous nature of the interaction: : When the client expects an immediate response from the service, it may become blocked while awaiting this response. This is direct, similar to when they call each other synchronously. Synchronous : In this case, the client doesn’t get blocked waiting for feedback. The response, if provided, might not be dispatched right away. This is similar to when they utilize a message broker. There’s a separate software with data channels. A service sends a notification there, and the services needing this information subscribe to it. They can then process this data asynchronously when possible. Asynchronous If, from a business logic perspective, something can be done asynchronously, it’s better to do it that way. This ensures greater stability and facilitates load balancing. STEP 5. TEST AND DEPLOY also has some specifics. In traditional monolithic structures, the entire application could be tested as a cohesive unit. In contrast, a microservices-based application may encompass numerous services that might not always be available for simultaneous testing. This interdependence makes end-to-end testing complex and demands alternative testing methods. Testing microservices There are specific testing types tailored to microservices: Unit testing, which focuses on individual service behavior Integration testing that ensures harmonious inter-service operation Performance testing for assessing system responsiveness Component testing for individual service functionalities Contract testing to examine user-service interactions End-to-end testing for holistic application performance. Each testing type ensures that microservices function individually and collectively to meet business objectives. Yet, testing microservices is not without its challenges. For instance, errors in one microservice can set off a chain reaction, complicating root cause analysis. With microservices often communicating through various channels and protocols, it demands specialized skills and knowledge. Furthermore, the necessity of testing multiple endpoints, coupled with the imperative for automated testing, requires proficiency in script writing and automated testing tools. Challenges and Best Practices of Microservices Development Microservices development requires a whole set of approaches to tackle typical challenges.  Awareness of these difficulties allows you to level them out at the early stages and prepare the environment for the successful implementation of microservices. DATA CONSISTENCY Ensuring accurate transactions and data consistency across different services can be challenging. While there’s no one-size-fits-all solution, there are some general guidelines for managing data within a microservices structure. For areas demanding robust consistency, one service can be the primary source of truth for a specific entity. Other services can access this primary source via an API. Some services might maintain their own version of the data or a portion of it. This data can be consistent with the primary data but isn’t viewed as the main source. For instance, in an e-commerce system, there might be a customer order service and a recommendation service. While the recommendation service could be privy to the order service’s events, in cases like a customer seeking a refund, it’s the order service that retains the comprehensive transaction record. Experienced developers can select the most suitable approaches to ensure data consistency, considering the nuances of the specific case. This is because there’s no universal way to achieve this seamlessly. TEAM ORGANIZATION While developing microservices, each team can operate with its own management, methodology, etc. Therefore, it’s essential to have a clear structure for synchronization and communication between teams. This is especially relevant if you plan to split the workload between in-house and outsourced engineers. The problem with highly formalized structures is that teams might efficiently create their own service, but if they don’t consider how other teams use that service, it can happen that the data provided by these services come in different formats, etc. Hence, it’s necessary for the project to have the role of an optimizer who coordinates the actions of the teams. Basically, we need to employ Conway’s Law here, which states that the architecture of software systems is similar to the communication structure of teams working on them. Therefore, if you want to create an architecture of autonomous services, you should first organize small autonomous development teams that can interact with each other effectively. DEVOPS EFFORTS Working with microservices requires DevOps efforts, as the infrastructure is more complex. With microservices, deployments must be coordinated to ensure all versions of the microservices can interact seamlessly, given they are all interconnected and might have backward incompatible changes. It’s crucial to prepare all dependencies in advance and employ flexible tooling (like Kubernetes and Docker). A DevOps engineer can help to launch and deploy this infrastructure. Microservice architecture troubleshooting also becomes more challenging as pinpointing the root cause of an error is tougher. In a monolithic structure, this process is relatively straightforward. With microservices, it’s more complicated since one service might receive data and then pass it to another, making it harder and more time-consuming to determine where in the chain things broke down. To address this, a centralized log aggregator, a deployment orchestration system, and a separate distributed tracing system need to be integrated, making the entire setup more manageable. Wrapping Up When migrating to microservices, a company needs to take into account many factors, including which parts of the system to bring to microservices, data management, establishing an effective infrastructure, and organizing the work of the team. Working with experienced engineers and solution architects is key here to achieve your goals in the best possible way. Also published here Author: Yevhen Kuzminov Ruby Team Leader/Solution Architect at MobiDev

How to Migrate from Monolith to Microservices: Challenges and Best Practices

How Product Owners Should Use Web Accessibility Testing

Augmented reality’s use and development have trended up over the past several years. This follows investments in mobile hardware capability, interest in immersive virtual experiences with the metaverse, and rising industry competition. Due to these variables, the augmented reality market is set to reach a value of $502 billion by 2027, growing at a CAGR of 62.7%, according to Research and Markets. If you’re planning to become part of these percentages with your augmented reality project, you should know about all the nuances of AR app development. In this guide, we’ll discuss the details of augmented reality development in 2022, including the choice of technologies and the development flow. Types of Augmented Reality Apps There are a number of different types of AR applications including marker-based, markerless, location-based, and superimposition AR solutions. MARKER-BASED AR These applications utilize a particular ‘marker’ like a QR code or other image. 3D content in the app is placed in the world relative to, or on top of the marker. An older, but interesting example of marker-based AR is the a gaming ‘peripheral’ that allows players to view a spell book on their screen. The book rotates and moves when they pick up the device and move it around. PlayStation 3 Wonderbook, The camera uses the patterns on the actual book as a reference to display the AR content, a technology that is often used today on Snapchat and Instagram. MARKERLESS AR Instead of using set patterns or codes to trigger the content, markerless AR utilizes a camera to detect environment patterns as well as motion sensors to detect surfaces and place 3D objects. This typically involves a number of different technologies working together, such as: GPS and other location tools Digital compass Camera Accelerometer and Gyroscope Depth sensors The latest devices are equipped with depth-sensing hardware (LiDAR, ToF) to improve precision. So markerless AR is powered not only by depth sensors and other positioning data but also by on top of this data. ML algorithms This allows for a more accurate rendering of 3D content and powers the illusion that digital objects are part of the real world. Apps like Pokemon Go utilize markerless AR. LOCATION-BASED AR When users enter a particular location, AR applications can use that data to accurately display virtual content. This is how location-based AR works. Instead of simply displaying an object in relative space, developers can show objects in geographical space for users to observe and interact with. Technology-wise, location-based AR relies on GPS, a digital compass, and an accelerometer. Moreover, there are several approaches to narrow down the position of the device: – BLE (Bluetooth low energy) beacons – VPS (Visual Positioning System) – Low-range Wi-Fi Direct – UWB (Ultra-WideBand) Superimposition AR This type of AR involves digitally replacing an object or superimposing a virtual object on top of another. For example, an app that can digitally change the color of your couch could be considered superimposition AR. This technique is useful for cause-and-effect demonstrations. For example, a user could point a camera at areas of their city to see what it looked like ten years ago from an archive of Google Maps. Technologies Used to Develop Augmented Reality Technologies used in augmented reality app development can depend on a number of factors, like the type of hardware being used, the available power of the device, and what application AR is being used for. Mobile Augmented Reality Platforms Smartphones have unique advantages compared to other AR platforms. They’re prevalent in the market and are extremely portable. This makes AR more accessible to many consumers since bulky headsets and elegant smart glasses haven’t quite hit the mainstream just yet. Because of this, mobile AR is a prime target for business applications. Although mobile AR may not be the most powerful or immersive, it certainly has the potential to be very profitable and is one of the most important to keep track of. Moreover, mobile AR can be a cost-effective way for business owners to join the metaverse trend. augmented reality trends Try-on solutions that allow you to test cosmetics or clothes before buying, AR avatars and filters available to users on a smartphone can help businesses to communicate with customers even in a virtual environment. There are three approaches toward mobile AR that businesses need to choose from: Native Android AR applications with ARCore Native iOS AR applications with ARKit Cross-platform apps Native augmented reality app development will allow developers to take advantage of the benefit of the power of a device. In turn, a cross-platform application may not be able to take advantage of powerful native features but minimize the development time. Creating the same app with native code on each platform will be more expensive, but if more power and features are required, it may be more effective. If the application is fairly simple, cross-platform code may be enough. Despite , developers on GitHub have historically seemed to prefer ARKit based on the number of repositories over the past several years. Android dominating the global OS market Year ARKit Repository Results ARCore Repository Results 2020 3,967 1,483 2021 4,504 1,874 2022 4,962 2,204 AR Development for iOS Devices: ARKit 5 and ARKit 6 Location Anchors Location anchors allow developers to affix virtual objects in the real world by using geographic coordinates. For example, location anchors could display a three-dimensional icon or text in space next to an iconic building. Since location anchors are dependent on Apple Maps data, this means that if the city is not supported by Apple Maps, functionality for location anchors may be limited. One of the most groundbreaking elements of location anchors is how location is approximated. GPS is simply not precise enough to provide location anchors on a user’s screen. To assist, developers can look around with their camera to allow the device to scan for features on buildings around them. By using these architectural features in conjunction with Apple Maps Look Around data, a user’s location can be better approximated for location anchors. Depth API Depth API is another valuable feature of ARKit 4 which continues to play an important role in ARKit 5 and ARKit 6. It utilizes one of the most powerful hardware features for AR on a mobile device, the scanner on the iPad Pro and Phone 12 Pro, 12 Pro Max, 13 Pro, 13 Pro Max, and 14 Pro smartphones. LiDAR This enables a much better analysis of scenes and allows for real-world objects to occlude virtual objects with much better accuracy. Raycasting API Apple’s Raycasting API enables enhanced object placement in conjunction with depth data. This allows for a much more accurate placement of virtual objects on a variety of surfaces while taking into account their curvature and angle. For example, it’s possible to use this to place an object on the side of a wall or along the curves of a sofa rather than simply flat on the floor. The LiDAR sensor allows for the scanning of surfaces to take place much more quickly compared to traditional methods. ARKit 6 Updates ARKit 6 is about to bring several new features to improve Some of the features that Apple is touting include better motion capture, improvements to camera access, and additional locations for Location Anchors. AR experiences on iOS devices. They also plan to include Plane Anchors, a feature that would allow tracking flat surfaces like tables. This makes it easier for those surfaces to be moved without disrupting the AR experience. ARKit 6 will be launched alongside iOS 16 this fall. As part of our ARKit research here at MobiDev, we tested using ARKit for gaze tracking. This opens up a number of possibilities for iOS, such as eye-based gestures, vision-based website heatmap analytics, and preventing distracted or drowsy driving. https://youtu.be/ktbLmbP5ZoA?embedable=true AR Development for Android Devices: ARCore In an effort to stay competitive, Google has pushed ARCore far to remain one of the most versatile AR development platforms in the world. Let’s explore some of the features that are used by developers. Cloud Anchors This tool allows users to place virtual objects in physical space that can be viewed by other users on their own devices. Google has made sure that Cloud Anchors can be seen by users on iOS devices as well. On the Android side, ARCore is introducing new features to match ARKit’s advances. introduces new Cloud Anchors endpoints and terrain anchors. These both improve the geographic anchoring of virtual objects. ARCore v1.33.0 Earlier this year, ARCore v.1.31.0 introduced ARCore Geospatial API, which, similar to ARKit Location Anchors, utilizes data from mapping databases. In this case, Google Earth and Street View image data is used to identify the user’s geographic location and display virtual objects in those locations accurately. The following MobiDev demo demonstrates how ARCore object detection features for a work. virtual user manual https://youtu.be/6ceN7YgSEdU?embedable=true AR Faces ARCore enables developers to work with high-quality facial renderings by generating a 468-point 3D model. Masks and filters can be applied after the user’s face is identified. This is one of the most popular use cases for AR app features. AR Images and advertising posters are just a few of the applications that are possible with augmented reality images. 2D markers can be used to implement these features as well as more advanced solutions like . Virtual business cards AR indoor navigation We have tested ARCore with indoor navigation solutions at MobiDev. Over the past several years, applications like indoor positioning have gotten even easier since our first experiments, making this technology more feasible for use in the real world. Check out this video to learn more about it. https://youtu.be/eQXKEnrHMxw?embedable=true ARKit vs ARCore Features Comparison These two frameworks for Android and iOS respectively are nearly identical when viewed from a features perspective. However, the real difference between these two devices is hardware consistency. Apple iPhone and iPad devices are largely more consistent when it comes to the behavior and capability of their hardware. Meanwhile, Android devices are built by a number of different manufacturers with different specifications. Because of this, it becomes more difficult to deliver a consistent experience across many different Android devices. Since Android hardware is less consistent, it’s important to keep in mind how powerful your AR experience will be and what devices it should be running on. Should it primarily be running on the latest and greatest Samsung and Pixel devices with high-performance AR features, or should it be less intensive to run on more devices? The choice is yours but experienced AR developers are always there to help you find the best solution. Cross-Platform Mobile AR Development in Unity If utilizing the full power of native AR frameworks on Android and iOS isn’t necessary and if your goal is faster time to market, cross-platform AR development in Unity may be a good option. is a helpful framework for cross-platform augmented reality app development. Despite not being able to take full advantage of each device, Unity AR Foundation is still fairly powerful. Unity AR Foundation It supports Plane detection, Anchors, Light estimation, 2D image tracking, 3D object tracking, body tracking, Occlusion, and more. However, there are some missing features depending on the platform you’re on. For example, Unity AR Foundation has limitations with some features for ARCore at the moment, such as 3D object tracking, meshing, 2D & 3D body tracking, collaborative participants, and human segmentation. If you want your app to be cross-platform, you’ll have to keep these missing features in mind. Web-Based Augmented Reality Technologies On one hand, web AR is an extremely accessible technology because it can run on a multitude of devices without installing any additional software. On the other hand, web AR is very limited in features and power. Some businesses are already utilizing web AR for technologies like . For example, Maybelline, L’Oréal, and other companies have the option for users to virtually try on cosmetic products using their front-facing camera and web AR software. virtual fitting room solutions Web AR is best implemented for simple tasks like facial recognition filters, changing the appearance or color of an object in a scene like hair, replacing backgrounds for videoconferencing, and more. It’s important to remember these limitations when deciding what platform your application should run on. Augmented Reality Development For AR Wearables When we talk about for augmented reality, we typically refer to gear like Microsoft HoloLens and more portable and comfortable glasses like Google Glass. wearable technology From a software engineering standpoint, Microsoft Hololence’s development is based on the Microsoft technology stack and Azure Cloud. Check our demo that showcases the capabilities of the Microsoft HoloLens for product presentation and educational purposes. https://youtu.be/vnVZhIRvyp0?embedable=true As for AR Glasses, most of the hardware is Android-based, and manufacturers provide SDKs for engineers to create apps. There are still some substantial aspects to be considered. The first is User Experience and User Interface, as the pattern of using such software is entirely different from what we are used to with smartphones. Then, engineers must be capable of building lightweight and optimized apps, as energy consumption is still the main pain point for many devices. Written by , Head of Mobile Development at . The full article was originally published and is based on MobiDev technology research. Andrew Makarov MobiDev here

Creating an Augmented Reality App: Technology Guide 2022

How Fintech Business Founders Can Develop Crypto Trading Apps

Cómo los fundadores de Fintech Business pueden desarrollar aplicaciones de intercambio de criptomonedas

Comment les fondateurs d'entreprises Fintech peuvent développer des applications de trading de crypto

फिनटेक बिजनेस फाउंडर्स क्रिप्टो ट्रेडिंग ऐप्स कैसे विकसित कर सकते हैं

フィンテック ビジネスの創業者が仮想通貨取引アプリを開発する方法

Como os fundadores de empresas fintech podem desenvolver aplicativos de negociação criptográfica

Làm thế nào những người sáng lập doanh nghiệp Fintech có thể phát triển ứng dụng giao dịch tiền điện tử

金融科技企业创始人如何开发加密货币交易应用程序

Workspace security can be a fiddly money drain, especially for corporations that deal with sensitive information or run multiple offices with thousands of employees. Electronic keys are one of the standard options for how security systems can be automated, but in reality, there are tons of downsides like lost, forgotten, or faked keys. Biometrics have shown solid alternatives to conventional security measures since they represent a concept of “what-you-are” authentication. This means a person can use their unique characteristics like fingerprint, iris, voice, or face to prove they have access to a certain space. Using biometrics as a method to authenticate guarantees the key can’t be lost, forgotten, or fabricated. So today we’ll speak about our experience with edge biometrics, which is the combination of edge devices, AI, and biometrics to implement AI security monitoring systems. What Are Edge Biometrics? First things first, what is ? In a traditional architecture of artificial intelligence, a common practice is to deploy models and data in the cloud, separate from the operating device or hardware sensor. This forces us to maintain the cloud server in a proper state, preserve a stable internet connection, and pay for the cloud service. If the storage appears inaccessible in case an internet connection is lost, the whole AI application becomes useless. edge AI “In contrast, the idea of edge AI is to deploy artificial intelligence applications right on the device, closer to the user. Edge devices may have their own GPU which allows us to process the input in place on the device. This provides a number of advantages such as reduced latency due to fact all actions are performed locally, on device. Overall cost and power consumption also become lower. Additionally, the system becomes portable, since the device can be easily moved from one location to another. Given the fact we don’t need large ecosystems, bandwidth requirements are also lower opposed to traditional security systems that rely on stable internet connection. Edge device can operate even in case of connection shut down, as data can be stored in device’s internal storage. Which makes the whole system design more reliable and robust.” Daniil Liadov Python engineer at MobiDev The only notable pitfall is that all the processing has to be done on the device in a short amount of time, and the hardware components need to be powerful enough and up to date for this function. For tasks like biometric authentication with face or voice recognition, fast response and reliability of the security system are critical. Since we want to ensure seamless user experience and proper security, relying on edge devices gives those benefits. Biometric information like employee faces and voices appears to be secure enough since they represent unique patterns that can be recognized by neural networks. Additionally, this type of data is easier to collect, as most businesses already have photos of their employees in their CRM or ERP. This way you can also avoid any privacy concerns by gathering, say, fingerprint samples of your people. Combined with edge, we can create a flexible AI security camera system for workspace entrance. So let’s discuss the implementation of such a system based on our own experience of automating office entrance processes with the help of edge biometrics. AI Surveillance System Design The main idea behind the project was to authenticate employees at the office entrance with just a glance in the camera. The computer vision model is able to identify a person’s face, compare it with the previously obtained photo and unlock the door. Voice verification was added as an additional step to avoid tricking the system in any way. The whole pipeline consists of 4 models that carry different tasks from face detection to speech recognition. All of these steps are done through a single-edge device that serves as a video/audio input sensor, as well as a controller for sending commands to lock/unlock doors. As an edge device, has been chosen. The main reasons behind this choice were the presence of GPU memory (which is vital for accelerating inference for deep learning projects) and the availability of Jetpack – SDK from NVIDIA, which allows coding on devices in Python3. Therefore, there is no strict need to convert the DS model to another format and nearly all of the codebase can be adapted to the device by DS engineers, so no rewriting from one programming language to another is required. NVIDIA Jetson Xavier So the general flow looks as follows: Input image is provided to the face detection model to find the user. Face identification model runs inference by extracting vectors and comparing them with the existing photo of an employee to identify whether it’s the same person. Another model is fed with voice samples to verify the voice of a specific person. Additionally, speech-to-text anti-spoofing is used as a technique to prevent any type of trickery. Let’s discuss each of the elements, and elaborate on the training and data collection procedure. DATASET GATHERING Before diving deep into system modules, it’s important to note what database is used. Our system relies on the availability of so-called reference, or ground-truth data for users. This data currently includes precomputed face and voice vectors for each user and looks like an array of numbers. The system also stores data from the successful attempts of logins for their possible use for re-training later. Given this, we’ve chosen the most lightweight solution, SQLite DB. With this DB, all the data is stored in a single file that’s easy to browse and back up, while data science engineers’ learning span is shorter. Since facial recognition requires photos of all employees that may enter the office, we used facial photos stored in corporate databases. A Jetson device placed on the office entrance also collected face data samples as people used face verification to open the door. Voice data wasn’t available initially, so we organized data gathering by asking people to record 20-second clips. Then, we used a voice verification model to obtain vectors for each person and stored them in DB. Voice samples can be collected with any audio input device. In our case, casual mobile phones and web cameras with inbuilt microphones were used to record voices. FACE DETECTION Face detection provides an answer to the question of whether there are any faces in a given scene. If there are, the model should give coordinates of each so you will know where each of the faces is located on the image, including facial landmarks. This info is important because we need to receive a face in a bounding box to run face identification on the next step. For face detection, we used the model with a MobileNet backbone from the InsightFace project. This model outputs four coordinates for each detected face on an image as well as 5 facial landmarks. The fact that images captured at different angles or with different optics can change the proportions of the face due to distortion. This may cause the model to struggle to identify the person. RetinaFace For this need, facial landmarks were used to perform warping, a technique to reduce possible differences between these images of the same person. As a result, obtained cropped and warped faces will look more similar, and extracted face vectors also will be more accurate. FACE IDENTIFICATION The next step is face identification. At this stage, the model has to identify the person from a given image, which is the obtained picture. Identification is done with the help of references (ground truth data). So here, the model will compare two vectors by measuring the distance score of the difference between two vectors to tell if it’s the same person standing before the camera. This assessment is compared with the initial photo of an employee we had. Face identification is done with the model with architecture. In order to make the model results more robust, before obtaining the face vector input, the image is averaged with the flipped version of itself. At this point, the flow of user identification looks as follows: SE-ResNet-50 VOICE VERIFICATION Next, we move to voice verification. This step should verify if two audios contain the voice of the same person or not. You may ask why not make allowance for voice identification as well? The answer is that facial recognition works much better than voice nowadays, and an image gives more information to recognize the user than a voice. To avoid identifying the same person as user A by face, and user B by voice, only face recognition is applied. The basic logic is almost the same as in the face identification stage, as we compared two vectors by the distance between them unless we found similar vectors. The only difference is that we already have a hypothesis about who is the person who is trying to pass from a previous face identification module. During the active development of the voice verification module, many issues were popping up. The previous model with Jasper architecture was not able to verify the recordings of the same person taken from different microphones. So we solved this problem by using architecture, which was trained on the dataset from the SpeechBrain framework which did a better job at verifying employees. ECAPA-TDNN VoxCeleb2 However, audio clips still needed some pre-processing. The goal was to improve the quality of audio recording by preserving voice and reducing present background noise. However, all the tested techniques affected the quality of the voice verification model badly. It is likely that even the slightest noise reduction could modify voice audio characteristics in recording so the model will not be able to correctly verify the person. Also, we performed an investigation into how long the audio recording should be and how many words the user should pronounce. As a result of this investigation, a few suggestions have been made. Such a recording should have a duration of at least 3 seconds and around 8 words need to be spoken. SPEECH-TO-TEXT ANTI-SPOOFING The final security grain was added with speech-to-text anti-spoofing built on from the Nemo framework. This model provides a decent quality user experience and is suitable for real-time scenarios. To measure how close what the person says to what the system expects, requires calculation of the Levenshtein distance between them. QuartzNet Obtaining a photo of an employee to trick the face verification module is an achievable task, as well as recording a voice sample. Speech-to-text anti-spoofing excludes scenarios in which an intruder tries to use a photo and audio of an authorized person to get inside the office. The idea is quite simple: when each person verifies themselves, they pronounce a phrase given by the system. The phrase consists of randomly selected words of a chosen set. While the number of words is not huge in a phrase, the actual number of possible combinations is. Applying random generation of phrases, we eliminate the possibility of spoofing the system, as it will require a large number of recorded phrases spoken by an authorized user. Having a photo of a user is not enough to trick an AI security system with this protection in place. The Benefits Of The Edge Biometrics System At this point, our edge biometric system follows a simple flow for a user, which requires them to p say a randomly generated phrase to unlock the door. Additionally, with face detection, we provide AI surveillance for the office entrance. “By adding multiple edge devices, the system can be scaled to different locations or easily modified. We can directly configure Jetson through the network, set up connection with low-level devices via GPIO interface, and upgrade it with new hardware quite easily, compared to a regular computer. We can also integrate with any digital security system that has a web API. But the main benefit here is that we can collect data for improving the system right from the device, since it appears convenient to gather data on the entrance, without any specific interruption.” Daniil Liadov Python engineer at MobiDev Written by , AI Engineer at Dmitriy Kisil MobiDev. The full article was originally published and is based on MobiDev technology research. here

Developing AI Security Systems With Edge Biometrics

Desarrollo de sistemas de seguridad de IA con Edge Biometrics

12 Augmented Reality Trends of 2023: New Milestones in Immersive Technology

For many years one of the main means of cancer diagnosis was a biopsy. A doctor would take a sample of the tissue in question, and a pathologist, among other tests, would make a thorough examination of it under the microscope. They would look for the affected cells, or for some other signs of developing the disease. That being said, this process is and was quite tedious, as the abnormal cells are minuscule compared to the enormous image of the tissue – the actual whole slide image. Modern histopathology involves multiple methods to analyze human cells, including tissue scans and anamnesis data. Today, healthcare organizations rely on or to save a digital scan of the entire histology result. WSI scans help doctors to analyze larger areas to detect malignant cells or other mutations in specific organ tissue. Whole Slide Imaging WSI However, the process of working with WSI is enormously time-consuming, since such slides usually have large resolutions. Running through them manually is a tough and time-consuming task. Given that this problem exists, the MobiDev team challenged themselves to search for an AI pathology solution capable of working with WSI and supporting digital healthcare workflows. Image credits to National Cancer Institute. What is WSI? WSI image is a high-resolution, large format scan of human tissue. WSI images are also called microscope slides, as they represent an enlarged image of cells and subcellular structure. Such slides are obtained by making multiple scans of different areas of a tissue sample and montaging them into one large image. As an output, we receive a high-quality digital scan that can be zoomed in up to 40 times without losing much detail. Compared to the traditional microscopic analysis of histology, WSI provides a number of advantages: Whole slide images usually come in giant resolutions like 100,000 x 100,000 pixels and bigger, which gives a lot of clarity when visually inspecting tissue areas. WSI is a digital scan that covers much larger areas all at once and is easier to zoom in/zoom out, compared to a microscopic view. The image can be sent to the healthcare experts as a digital copy. Additionally, WSI scans can’t deteriorate like specimens on a glass slide. However, in terms of operability, WSI analysis is not so great due to the time and effort needed during the manual examination. In real-world practice, a doctor would sit before the image scanning by eye, zooming in and out for hours. In the case of manual examination, a whole slide should be examined to spot suspicious cell structures or mutations, because skipping some areas may result in medical error. This is a general problem for healthcare workflows since long processes prevent quick diagnosis and efficient treatment, which are critical for patients with cancer and for histopathology as a whole. With the advances in digital pathology and fluctuations in healthcare regulatory requirements, the application of artificial intelligence became possible to use to process clinical data. Given the fact that WSI is a gigapixel-sized image document that contains tons of unstructured data, the problem with manual examination can be addressed through machine learning. Applying AI for WSI scans processing Independent of the content in the image, WSI allows for the image to be broken down into features which are used for training machine learning models that are capable of running object detection, recognition, and classification tasks. These tasks are handled with computer vision that relies on optical sensors or digitally processes images to extract meaningful information. However, since we don’t know where the potential tumor cells are, we can’t perform one essential ML task: data labelling. Labels are required to tell the model what these or those cells are for it to learn the difference between normal and damaged tissue. To solve this, convolutional neural networks, or CNNs, are used. CNN requires little pre-processing of data input, as they are mainly used for visual data to find patterns autonomously. Applied to WSI scans, CNN is capable of extracting features out of weakly labelled, or completely unlabeled data. Extracted features then can be used to represent the whole slide in compact form, which can be helpful for image storing or grouping similar items. However, minor pre-processing steps are still required, since it’s too CPU-demanding for the model to work with the whole slide. Due to the giant image size, the image is most likely to be sliced into equally-sized square images. So now, let’s look at the example in practice. MobiDev’s Research: WSI processing through CLAM approach With their extensive expertise in computer vision, Mobidev approached the WSI processing system design through clustering. This is a constrained attention multiple instance learning ( ) approach. As a training dataset, publicly available Clear Cell Renal Cell Carcinoma was used. CLAM dataset To prepare data for training, images need to be cut into patches (equally sized square images), because the initial image size is too large. Also, the data used for the experiment was weakly labeled, meaning there were no labels for each patch of images. The task was to create a model capable of analyzing WSI scans and then highlight regions of interest, or in other words, suspicious cells, found in the tissue. As long as healthcare is based on critical decisions, the role of AI in pathology is to support the doctor with hints to speed up the diagnostic process. However, the prediction should also be descriptive, as long as the user has to understand how the model derived this or that result. This is what’s called AI explainability as a feature of modern AI pathology systems. You can read more about the forms and approaches in AI explainability in decision-making applications. CLAM is a data-efficient approach to working with weakly-labeled whole-slide imagery. The essence of it can be broken down into a number of steps: On the WSI, the tissue/cell sample is segmented from the background using computer vision algorithms. Within the segmented area, the tissue is split in equally-sized square patches (usually 256px). Patches are encoded once by a pretrained CNN into a descriptive feature representation. During training and inference, extracted patches in each WSI are passed to a CLAM model as feature vectors. For each class, the attention network ranks each region in the slide and assigns an attention score based on its relative importance to the slide-level diagnosis. Attention-pooling weighs patches by their respective attention scores and summarizes patch-level features into slide-level representations, which are used to make the final diagnostic prediction. The attention scores can be visualized as a heatmap to identify ROIs and interpret the important features. Left: direct visualization of the attention scores. Each pixel represents a 256px patch. Right: attention scores blended onto the WSI. The colored regions represent the most attended parts of the sample. As we can see, the CLAM’s attention network is quite interpretable. It outputs “importance scores” for each patch, so those can be visualized as a heatmap. If you take a closer look at the WSI on the right, you may see the affected cells colored blue, meaning the network determined they would affect the prediction the most. Using neural networks to process weakly labeled WSI images provides a couple of advantages in this case. First of all, we can discover clinically relevant, but previously unrecognized morphological characteristics that have not been used in visual assessment by pathologists. Secondly, the model can recognize features that are beyond human perception, which can potentially provide more accurate diagnostic results and have a positive impact for surgical or therapeutic results. Addressing the WSI complexity and interpretability This approach addresses important WSI processing difficulties. For example, to handle the large size of the slide, we segment the actual tissue from the scan background, trimming unnecessary information. Then, we split the remaining ROI into tiles that are treated equally to fit them into the GPU. Also, another difficulty is labeling. It takes hours for a pathologist to look through the slide and diagnose, let alone label it, i.e. identifying affected cells. On the contrary, CLAM is designed to work with this kind of weakly-labeled data, where only the slide-level labels are known (i.e. diagnosis.) CLAM itself determines which regions of the tissue affect the predicted diagnosis. Written by , AI Engineer at . Anton Kantsemal MobiDev The full article was originally published and is based on MobiDev technology research. here

How AI Is Helping To Diagnose Cancer

As the American Society of Quality , many organizations have quality-related costs of up to 40% of their total production revenue. A large part of this cost comes from the inefficiency of manual inspection, which is the most common way to provide quality control in manufacturing. reports The application of artificial intelligence for quality control automation presents a more productive and accurate way of doing a visual inspection of production lines. However, traditional machine learning methods present several limitations to how we can train and utilize models for defect detection. So in this article, we’ll discuss the advantages of unsupervised learning for defect detection, and elaborate on the approaches MobiDev uses in our practical experience. What is AI defect detection and where it is used? is based on computer vision that provides capabilities for automating the whole process using machine learning algorithms. Defect detection models are trained to visually examine items that pass the production line and recognize anomalies on their surface, and spot inconsistencies in dimensions, shape, or color. AI defect detection AI quality inspection The output depends on what the model is trained to do, but in the case of defect detection, the flow typically looks like this: https://www.youtube.com/watch?v=UY6xbrcViVw Applied to quality control processes, defect detection AI is efficient at inspecting large production lines and spotting faults even on the smallest parts of a final product. This relates to a large spectrum of manufactured products that may contain surface defects of different natures. Intel describes a case of implementing computer vision to automate tire quality inspection. As is stated in the , the accuracy of quality control rose up from 90% to 99% percent, while approximately $49,000 in labor costs were cut by the production line. report But such systems are not bound to stationary hardware in the factory. For instance, drones with cameras can be used for inspecting or other outdoor surfaces, which significantly decreases the time required for covering large city areas. pavement defects The pharmacy industry also benefits from inspecting manufacturing lines for different products. For instance, Orobix applies defect detection to with a specific type of camera that can be used by an untrained human operator. The same principle is applied to inspecting defects, like cracks and air blobs caught in the glass. drug manufacturing pharmaceutical glass Such examples can be found in the food industry, textile, electronics, heavy manufacturing, and other branches. But, there are some specific problems in how we can approach defect detection algorithms with traditional machine learning. As manufacturers inspect thousands of products daily, it becomes difficult to collect sample data for training, and also to label it. This is where unsupervised learning comes into play. What is unsupervised learning? The majority of machine learning applications rely on methods. Supervised learning entails that we provide ground truth information to the model by manually labeling collected data. supervised machine learning In terms of a production line, collecting and labeling data can be impossible, since there is no way we can gather all the variations of cracks or dents on a product to ensure accurate detection by the model. Here we face four problems: difficulties with obtaining a large amount of anomalous data possibility of a very small difference between a normal and an anomalous sample a significant difference between two anomalous samples inability to know in advance the type and number of anomalies algorithms allow you to find patterns in a data set without pre-labeled results and discover the underlying structure of the data where it is impossible to train the algorithm the way you normally would. Unsupervised machine learning Unlike supervised learning, the flow of training becomes less labor intensive, as we expect the model to discover patterns in data with a higher threshold for variations. Anomaly detection reveals previously unseen rare objects or events without any prior knowledge about them. The only available information is that the percentage of anomalies in the dataset is small. Concerning defect detection, this helps to resolve the problem with data labeling and collecting huge amounts of samples. So let’s see how unsupervised learning methods can be used for defect detection model training. How does unsupervised learning apply to defect detection? Defect detection relates to the problem of anomaly detection in machine learning. While we don’t rely on labeling, there are other approaches in unsupervised learning that aim at grouping data and providing hints to the model. is grouping unlabeled examples according to similarity.  Clustering is widely used for recommender engines, market or customer segmentation, social network analysis, or search result clustering. Clustering aims to observe frequently occurring patterns, correlations, or associations from datasets. Association mining are designed to model the distribution probability with latent variables. It is mainly used for data preprocessing, reduction of features in a dataset, or decomposing the dataset into multiple components based on features. Latent variable models Uncovered patterns with unsupervised learning can be used for implementing traditional machine learning models. For instance, we might apply clustering to the available data, and then use those clusters as a training dataset for supervised learning models. Concrete Crack Detection with Unsupervised ML Having a vast experience in machine learning, we’ve conducted an experiment using the . The goal was to create a model capable of recognizing images with defects and normal ones using unsupervised learning. Concrete Crack dataset Additionally, the study checks how the number of defect images affects certain algorithms used in this project. In the use case we selected, we assume that image labels cannot be known in advance during training. Only a test dataset is labeled in order to verify the quality of model prediction, as training occurs by an unsupervised approach. So here we used five different approaches to get classification results from an unsupervised learning model. CLUSTERING Since we don’t have any labeled ground truth data, grouping unlabeled examples are done with clustering. In our case, there are two clusters of images we need to single out from the dataset. This was performed with a pre-trained VGG16 convolutional neural network for feature extraction and K-means for clustering. What clustering did here, is group images with and without cracks based on their visual similarities. In a nutshell, clustering looks something like this: Clustering methods are easy to implement and usually are considered as a baseline approach for further deep learning modeling. BIRCH CLUSTERING Upon this approach, images were clustered based on visual similarity with a pre-trained ResNet50 neural network for feature extraction and for clustering. This algorithm constructs a tree data structure with the cluster centroids being read off the leaf. Birch It is a memory-efficient, online-learning algorithm. The results of clustering were visualized with : Principal Component Analysis As we can see, birch clustering shows a pretty good distribution of classes, even on points where the sample is quite far from its centroid. CUSTOM CONVOLUTIONAL AUTOENCODER Custom convolutional autoencoder contains two blocks: encoder and decoder. It helps to obtain features in the encoder part and reconstruct images from them in the decoder part. As we don’t have labels for network training, we need to choose another approach to obtain classes – for example, an adaptively selectable threshold. The purpose of an adaptively selectable threshold is to divide two distributions (crack-free images and images with cracks) as precisely as possible: DCGAN DCGAN generates images from z-space with the help of adversarial loss (BCALoss). Finally, we have three losses – generator loss, discriminator loss, and MSE loss (to compare generated images and ground truth). We can build our classification on the same approach as in custom autoencoder – through comparison between losses on images with cracks and without with the help of an adaptive selectable threshold. For threshold, it will be appropriate to use discriminator loss or MSE loss depending on their distributions. GANOMALY uses the conditional GAN approach to train a Generator to produce images of the normal data. During inference, when an anomalous image is passed, it is not able to capture the data correctly. GANomaly It leads to poor reconstruction for defected images and good reconstruction for normal ones and gives the anomaly score. How to approach unsupervised anomaly detection Perhaps, the most beneficial side of unsupervised learning techniques is that we can avoid gathering tremendous amounts of sample data, and labeling it for training. Applying unsupervised learning techniques to derive data patterns, we’re not limited to which model can be used for actual classification and defect detection. However, unsupervised learning models are suited better for segmenting existing data into classes, since it’s quite difficult to check the model prediction accuracy, especially without a labeled dataset. Written by , AI/ML Team Leader at . Viktoriia Akhremenko MobiDev The full article was originally published and is based on MobiDev technology research. here

Stories, News & Mentions about MOBIDEV

Mobidev’s Stories on HackerNoon

Mobidev's News & Mentions