How to Use OpenCV in React Native for Image Processing

If you have ever wondered how to process your images using OpenCV with React Native, then you’re in the right place. OpenCV together with enables you to (most likely you’d like to process images taken by your device’s camera). React Native process images on mobile devices The top of those are: advantages Easy to implement. Easy to use. Lots of tutorials all over the Internet and solid official documentation of OpenCV. The size of your mobile app will be only a dozen or so megabytes bigger. Let me show you how I used the OpenCV and React Native step by step to process my images, but first a few words of introduction. What is OpenCV? OpenCV is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. (Open Source Computer Vision Library) The library has more than , including a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. 2500 optimized algorithms These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc. OpenCV has more than and an estimated number of . 47k people community downloads exceeding 14 million The library is used extensively in companies, research groups and by governmental bodies. OpenCV is written natively in C++. Since 2010 OpenCV was ported to the Android environment, it allows using the full power of the library in mobile applications development. In 2012 OpenCV development team actively worked on adding extended support for iOS. Full integration is available since version 2.4.2 (2012). was first . React Native lets you using only JavaScript. It uses the same design as React, letting you compose a rich mobile UI from declarative components. React Native released in 2015 by Facebook build mobile apps React Native and OpenCV are good friends! If we google for “react native opencv” we stumble upon the following links: ‘ ’As the first words in the docs say: “Currently being developed 🙂 React Native Bindings for OpenCV. Yet there are no React Native bindings.” Also it’s incomplete and uploading 500+MB of the OpenCV library is not a good idea. GitHub — ma-pe/react-native-opencv: OpenCV Bindings for React … ‘ ’An unanswered question about React Native and OpenCV. Can i use openCV in react native ?? · Issue #88 · wix/react-native … ‘ ’A problem with linking libraries with no indication on how to do it in the first place. opencv/opencv — GitHub ‘ ’Answers are that it’s possible but with no example of how to do it. (Newbie) React Native with OpenCV : reactnative — Reddit What is this tutorial about? In this tutorial, we will be building an example project which to take a photo, and if the taken image is blurred or clear. uses the device’s camera processes it with native code returns the information Doing it in plain JavaScript would be highly ineffective. JavaScript is not sufficient for very heavy calculations. Take a note that we are by no means Java or Objective-C developers hence our Java/Objective-C code may be far from good. If you want to contribute to this subject, please contact us. OpenCV basic preparation Step 1: react-native init reactNativeOpenCvTutorial Run script in your project’s catalog ( which downloads and inserts openCV files for both Android and iOS. The paths in the file might not match your preferences so you may need to change them. Step 2: downloadAndInsertOpenCV.sh can be found here) Tutorial for Android Open your project in Android Studio. Step 1: Follow the tips in Android Studio for synchronizing your project. Step 2: Download . In my case, it’s 3.4.1. Step 3: the newest version of OpenCV for Android Import OpenCV to Android Studio: From , choose folder in the unzipped OpenCV archive. Step 4: File -> New -> Import Module sdk/java Update under imported OpenCV module to update 4 fields to match your project’s : Step 5: build.gradle build.gradle compileSdkVersion buildToolsVersion minSdkVersion targetSdkVersion. Add module dependency: Step 6: , and select the tab. Click icon at the bottom, choose and select the imported OpenCV module. Application -> Module Settings Dependencies + Module Dependency For Android Studio v1.2.2, to access in the project view, right-click the dependent module and choose . Module Settings Open Module Settings Open up . Module Settings Click on the , select and select the OpenCV library from the list. + Module Dependency Inside create a package called i.e. . Step 7: android/app/src/java com.reactlibrary Update your Manifest with proper permissions: Step 8: See the . whole file Create a file inside your newly created package and . Step 9: RNOpenCvLibraryModule.java fill it as shown here Create a file called inside your newly created package and . Step 10: RNOpenCvLibraryPackage.java fill it as shown here Add proper imports in your file, add your OpenCV package to the list and proper code to the class like this: Step 11: MainApplication.java MainApplication Imports: import com.reactlibrary.RNOpenCvLibraryPackage;import org.opencv.android.BaseLoaderCallback;import org.opencv.android.LoaderCallbackInterface;import org.opencv.android.OpenCVLoader;import android.util.Log; Add to your list of packages. new RNOpenCvLibraryPackage() Add in your class: BaseLoaderCallback MainApplication Also, add following callbacks to your class: MainApplication See the . whole file here Tutorial for iOS Open the iOS project in XCode. Step 1: Add to your . Step 2: opencv2.framework Linked Frameworks and Libraries Create a new group in the iOS catalog. I named it “OpenCV”. Step 3: Add a .pch file and insert it into the OpenCV catalog. Step 4: Add proper content to your file — . Step 5: .pch shown here Create a file named and . Step 6: RNOpenCvLibrary.h fill it as shown here Create a file named and . Step 7: RNOpenCvLibrary.mm fill it as shown here Set to and set like this: Step 8: Precompile Prefix Header Yes Prefix Header path Add the following to your file: Step 9: Info.plist NSCameraUsageDescription Your message to user when the camera is accessed for the first time NSPhotoLibraryAddUsageDescription Your message to user when the photo library is accessed for the first time See the . whole file Final part — JavaScript Inside your src folder create a folder named e.g. and a file named and fill it with: Step 1: NativeModules OpenCV.js, import { NativeModules } from ‘react-native’; export default NativeModules.RNOpenCvLibrary; See the . whole file We will be using some third-party libraries for quick setup. Open up the terminal and type: Step 2: npm i –save react-native-svgnpm i –save react-native-cameranpm i –save react-native-easy-toast Don’t forget to link the libraries: . Step 3: react-native link We will be using as a reference. this file In line 126 we set up the camera and in line 135 we create a touchable element to handle taking photos. Taking photos is handled by the takePicture function. It takes a photo, saves the data in the local state and proceeds to check if the photo is blurred. The function uses native function and returns a simple information if the photo is blurred or not. proceedWithCheckingBlurryImage checkForBlurryImage Note: if you encounter build problems for Android connected to the camera, refer to my , especially line and resync the project in Android Studio. Step 4: build.gradle file maven { url “[https://jitpack.io](https://jitpack.io)” } If you have errors related to Google Play services, check this . issue on GitHub If you have errors related to react-native-camera, check the master branch by changing the version in like this: package.json “react-native-camera”: “git+ " https://git@github.com/react-native-community/react-native-camera Example results An example clear photo. We receive a preview of the photo and have the option to either use it or repeat it. An example blurred photo. We receive a toast message informing us that the photo is blurred and have to repeat it. Don’t forget to see the final product on our GitHub ! References: (StackOverflow) Setting up OpenCV for Android in Android Studio (Blog post) Setting up OpenCV for iOS in XCode (GitHub) Native modules for React Native (OpenCV official) Native Java code for checking if provided image is blurry (StackOverflow) Native Objective-C code for checking if provided image is blurry (OpenCV official) OpenCV’s official website This was published on Brainhub’s blog here . See our other React Native stories like ‘ ’! How to deliver a React Native app to the client Looking for help to build your React Native or React application? Drop us a line .