637 reads

From TF to TFLite: Deploying ML Models on Mobile [Part 2]

by Nagarakshitha RamuJuly 16th, 2020

Too Long; Didn't Read

This is part 2 of the two-part article on deploying ML models on mobile. We saw how to convert our ML models to TfLite format here. The Android app we are building has a GAN model generating handwritten digits and a classifier model predicting the generated digit. Find the code for the android app here. To load the models in Android, we must import and use the model to load and use Tflite interpreter. The model can be loaded as follows: create an Interpreter object.

Company Mentioned

featured image - From TF to TFLite: Deploying ML Models on Mobile [Part 2]

This is part 2 of the two-part article on deploying ML models on mobile. We saw how to convert our ML models to TfLite format here. For those of you who came here first, I recommend you click on the above link to get the whole picture. If you just want the android part ,the demo app we are building has a GAN model generating handwritten digits and a classifier model predicting the generated digit.

Tl;dr : Find the code for the android app here.

Part 2: The Android Story

Now that we have the TfLite model ready along with information about inputs and outputs to the model, we need to add it to our Android Studio project.

Step 1 : Create a new Android Studio Project.

Step 2 : Go to the location

Project -> app -> src -> main

,in your android project folder and create a directory called

assets

Step 3 : Paste your .tflite file(s) in the assets directory. The structure will look something like this -

Let us proceed to adding dependencies in android studio.. Add the following in

buid.gradle(Module:app)

android{
aaptOptions {
        noCompress "tflite"
        noCompress "lite"
    }
}

dependencies {
implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'
}

To load the models in Android, we must import and use TfLite interpreter.
TfLite Interpreter is used for the following:

Loading the model
Transform data
Running inference
Interpreting output

The model can be loaded as follows:

Create an Interpreter object and use the function loadModelFile() to load the TfLite model as a MappedByteBuffer. Here, the

MODEL_FILE

holds the name of your model as saved in the assets folder.

Interpreter tflite_gan;
String gan_model="gan_model.tflite";

 try {
            tflite_gan = new Interpreter(loadModelFile(context, gan_model));
        }
        catch (IOException e) {
            e.printStackTrace();
        }

    }
private MappedByteBuffer loadModelFile(Context c, String MODEL_FILE) throws IOException {
        AssetFileDescriptor fileDescriptor = c.getAssets().openFd(MODEL_FILE);
        FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
        FileChannel fileChannel = inputStream.getChannel();
        long startOffset = fileDescriptor.getStartOffset();
        long declaredLength = fileDescriptor.getDeclaredLength();
        return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
    }

For the GAN model, let us take the input and output ByteBuffers for easy pixel manipulation for image display.

First let us generate random latent points of a float array of [1x100] to provide as input ,as shown in the model details in part 1 of the article.

private float[][] generateLatentPoints(int n, int dim){
        Random rn = new Random();
        float[][] gan_input=new float[1][100];
        for(int i =0; i<100; i ++){

            float random_number=rn.nextFloat();
            gan_input[0][i]=random_number;
        }
        return gan_input;
    }

Next, we will copy this data into a ByteBuffer and also allocate a ByteBuffer to store the output. As discussed, the size allocation is based on the model information. The last line in the code runs the ML model

 float[][] input = generateLatentPoints(N,DIM);
        ByteBuffer input_data = ByteBuffer.allocate( 100 * BYTE_SIZE_OF_FLOAT);
        input_data.order(ByteOrder.nativeOrder());

        ByteBuffer gan_output = ByteBuffer.allocate(1 *28 * 28 * 1 * BYTE_SIZE_OF_FLOAT);
        gan_output.order(ByteOrder.nativeOrder());

        input_data.clear();
        gan_output.clear();
        input_data.rewind();

        for (int i=0;i<100;i++) {
            input_data.putFloat(input[0][i]);
        }

        tflite_gan.run(input, gan_output);

In the above code,

tflite_gan.run(input,output)

takes the input and gives output as a ByteBuffer which we shall render as an image. The output is essentially an array of [1x28x28x1]. The following snippet of code can be used to generate a Bitmap.

 gan_output.rewind();
        Bitmap bitmap = Bitmap.createBitmap(IMAGE_WIDTH, IMAGE_HEIGHT, Bitmap.Config.RGB_565);
        int [] pixels = new int[IMAGE_WIDTH * IMAGE_HEIGHT];
        for (int i = 0; i < IMAGE_WIDTH*IMAGE_HEIGHT; i++) {

            int a = 0xFF;
            float r = gan_output.getFloat() * 255.0f;
            pixels[i] = a << 24 |  (((int) r)<<8);
        }
        //This is a colored image which you can display
        bitmap.setPixels(pixels, 0, 28, 0, 0, IMAGE_WIDTH, IMAGE_HEIGHT);

All you have to do is set this image to an ImageView or wherever you want to display it on your app.

Yay! you just deployed a GAN model on android.

Let us proceed to MNIST classifier model.

The process remains same for loading and running the TfLite model. Replace the MODEL_NAME with the classifier model name. The input to the classifier is the output from GAN, which is a 28x28 image. We have to process this image in Black and White which the classifier is trained to identify. This can be done by following -

//Create a Black & white image by taking HSV values from RGB Image
        Bitmap bwBitmap = Bitmap.createBitmap( bitmap.getWidth(), bitmap.getHeight(), Bitmap.Config.RGB_565 );
        float[] hsv = new float[ 3 ];
        for( int col = 0; col < bitmap.getWidth(); col++ ) {
            for( int row = 0; row < bitmap.getHeight(); row++ ) {
                Color.colorToHSV( bitmap.getPixel( col, row ), hsv );
                if( hsv[ 2 ] > 0.5f ) {
                    bwBitmap.setPixel( col, row, 0xff000000 );
                } else {
                    bwBitmap.setPixel( col, row, 0xffffffff );

                }
            }
        }

        return bwBitmap;

Again, we shall provide input as a BYteBuffer. This has to be processed. Let us do so by setting 0 for white and 255 for black pixels.

public ByteBuffer Process(Bitmap bitmap){

        ByteBuffer inputBuffer = ByteBuffer.allocateDirect(BYTE_SIZE_OF_FLOAT * DIM_BATCH_SIZE * DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE);
        inputBuffer.order(ByteOrder.nativeOrder());
        int[] pixels = new int[28 * 28];
        // Load bitmap pixels into the temporary pixels variable
        bitmap.getPixels(pixels, 0, 28, 0, 0, 28, 28);
        for (int i = 0; i < pixels.length; ++i) {
            // Set 0 for white and 255 for black pixels
            int pixel = pixels[i];
            int channel = pixel & 0xff;
            inputBuffer.putFloat(0xff - channel);
        }
        return inputBuffer;

    }

We have reached the last step of the process - getting the predicted digit from the classifier based on the most probable value. The output contains a 2D array with index against a probability.Iterating through it, we get the most probable value and that index will be our answer. Since we know that the output is a [1x10] float array, we will save it in the same form and iterate through it.

public float Classify(ByteBuffer input){

        float[][] mnistOutput = new float[DIM_BATCH_SIZE][NUMBER_LENGTH];
        tflite_mnist.run(input, mnistOutput);
        float min_probability= Float.MIN_VALUE;
        float digit = 0;
        for (int i = 0; i < mnistOutput[0].length; i++) {
            float value = mnistOutput[0][i];
            if(value>min_probability){
                digit=i;
            }

    }
        return digit;

    }

That digit returned right there is your predicted output!

You can deploy as many models as you like in the same way. The key is to know and understand the input and outputs required by each of them and you are good to go. We can also get multiple outputs from the Interpreter using the following piece of code.

import org.tensorflow.lite.Interpreter;
Interpreter tflite;
tflite.runForMultipleInputsOutputs(inputs,outputs)

Make sure the data structure used to save your outputs is able to accommodate multiple arrays. For instance, you can define the output as a HashMap of arrays as shown.

output1 = new float[1][28][28][1];
output2 = new float[1][14][14][34];
Map<Integer, Object> outputs = new HashMap<>();
outputs.put(0, out1);
outputs.put(1, out2);