Machine Learning for Android Developers with the Mobile Vision API — Part 3 — Text Detection

Incase you missed it, here are the prequels to this article about the Mobile Vision API. was on the Face Detection API while was on the Barcode Detection API. The first post the second Text Detection API According to , the Text Detection API allows for detecting text in images and videos and it breaks down those texts into blocks (paragraphs/columns), lines (sets of words on the same vertical axis) and words (set of alphanumeric characters on the same vertical axis). The API recognizes text in various Latin based languages. the overview Potential applications I’ll write about what’s possible with this API before I go ahead to explain how to use it . Imagine you were invited to attend the Firebase Dev Summit in Berlin and you got all excited, but you didn’t know a word of German. The only foreign language you understand (apart from English) is some Spanish you picked up on two years ago. Tough huh? How would you communicate if all the signs were in German? Typing the text into Google Translate ALL THE TIME was going to be out of the question because you know German words have a reputation for being unusually long. An option would be to have some sort of way to detect any text you want translated and be able to receive translations in your preferred language immediately. The Text Detection API will help with detecting the text but the Google Translate API will be used for the translations. Duolingo Another possibility is converting a very large amount of text (e.g. from a book) into digital format. The traditional method would be to scan each page, which might damage the . With the Text Detection API, all that’ll be needed is a device with a camera for focusing on the text and maybe some API that uploads the recognized text to a server. book Getting started Here, we are going to detect text from a default image preloaded in an app using the Text Detection API. I initially wanted to take this a step further by translating that text into a specified language as described in the scenario above but I left that part out when I discovered the Translate API is billed per usage. Google Here we go (again)… Create a new project in Android Studio. Import Google Play Services SDK for the Mobile Vision API into your app level build.gradle file. As at the time of writing this article, the latest version is 9.8.0. You are bound to hit the 65k method limit if you import the whole SDK instead of the specific one (play-services-vision) you need. compile 'com.google.android.gms:play-services-vision:9.8.0' To enable the vision dependencies to be automatically installed for text detection, add this meta-data to the manifest file. This ensures the appropriate libraries are downloaded in time for first time users of the app. ocr We’ll create a very simple layout consisting of a , an and a . The loads a cute image of a cat with some German text from the drawable folder. The button starts the processing of the image while the displays whatever text is detected from the image. Button ImageView TextView ImageView TextView Just like the Face Detection and Barcode Detection API, the image has to be converted into a to be processed. Bitmap Bitmap textBitmap = BitmapFactory. (getResources(), R.drawable.cute_cat_image); decodeResource The Text Recognizer is initialized to process the image already present in the . ImageView TextRecognizer textRecognizer = new TextRecognizer.Builder(this).build(); Next, we need to check if the text recognizer is operational already. There’s always the possibility that it won’t work the first time because a library needs to be downloaded to the device and it might not have been completed in time for use. if (!textRecognizer.isOperational()) {new AlertDialog.Builder(this).setMessage("Text recognizer could not be set up on your device :(").show();return;} We then create a frame using the text bitmap and call the text recognizer. Frame frame = new Frame.Builder().setBitmap(textBitmap).build();SparseArray text = textRecognizer.detect(frame); Our results are now contained in the SparseArray which is then displayed in the . TextView for (int i = 0; i < text.size(); i++) {TextBlock textBlock = text.valueAt(i);if (textBlock != null && textBlock.getValue() != null) {detectedText += textBlock.getValue();}}detectedTextView.setText(detectedText); The Text recognizer should be released once the transaction is completed. textRecognizer.release(); Please note that all these operations should be carried out on a background thread. The code from this article is . on Github here References: _Text recognition is the process of detecting text in images and video streams and recognizing the text contained…_developers.google.com Text Recognition API Overview | Mobile Vision | Google Developers _Optical Character Recognition (OCR) gives a computer the ability to read text that appears in an image, letting…_codelabs.developers.google.com See and Understand Text using OCR with Mobile Vision Text API for Android _text-detector - This application contains all the code from my article on the Text Detector API._github.com moyheen/text-detector Thought this was great? Please don’t forget to “Recommend” and “Share”.