Tensorflow Vs Keras? — Comparison by building a model for image classification.

Yes , as the title says , it has been very usual talk among data-scientists (even you!) where a few say , TensorFlow is better and some say Keras is way good! Let’s see how this thing actually works out in practice in the case of image classification. Before that let’s introduce these two terms Keras and Tensorflow and help you build a powerful image classifier within 10 min! Tensorflow: Tensorflow is the most used library to develop models in deep learning. It has been the best ever library which has been completely opted by many geeks in their daily experiments . Could you imagine if I say that Google has put Tensor Processing Units (TPU) just to deal with tensors ? Yes, they have. They have put a separate class of instances called TPU which has the most power driven computational power to handle the deep learning models of tensorflow. Time to BUILD IT! I’ll now help you create a powerful image classifier using tensorflow. Wait! what is a classifier? It’s just a simple question you throw to your tensorflow code asking whether the given image is a rose or a tulip. So , first things first.Let us install tensorflow on the machine. There are two versions proivided by the official documentation i.e., CPU and the GPU version. For CPU version : pip tensorflow install And please note , I am writing this blog after experimenting on a GPU and NOT CPU. GPU installation is neatly given . here Now , let us take the Google’s Tensorflow for poets experiment to train a model. This repository of Google has amazing scripts for easy experiments on images. It is very much concise and sufficient for our purpose. Remember the word which I used before? Yes , that word comes into action when we use something called Transfer learning is a powerful way of using the pre-trained models which have been trained for days or might be weeks and then changing the final layer to adjust to our own set of classes. powerful transfer learning. Inception V3 is a very good model which has been ranked 2nd in for image classification. It has been mentioned as the best network for transfer learning for datasets with less number of images per class. 2015 ImageNet Challenge (Inception V3) Now clone the git repository: git clone http //github. /googlecodelabs/tensorflow- -poets- tensorflow- -poets- s: com for 2 cd for 2 Now , you get to choose your images . All you have to do is put the dataset folder in the below fashion. — Dataset folder - / — image — image / — image — image class 1 1 2 class 2 1 2 (FLOWER DATA) It should look something like above (Ignore the image.py). I have got the above folder by: flower_photos curl http: download.tensorflow.org flower_photos.tgz | tar xz -C tf_files // /example_images/ Creating the Dataset You can use whatever images you’d like. The more the better (aim for a few thousand). Separate them by categories as done above, and make sure they are in a folder called . tf_files You can download pre-exiting datasets of various use cases like cancer detection to characters in Game of Thrones. Here is various image classification datasets. Or if you have your unique use case, you can create your very own dataset for it. You can download images from the web and to make a big dataset in no time, use an annotation tool like , where you upload the images and tag images manually in a ziffy. Better yet, the output from Dataturks can be easily used to building the . Dataturks tf_files (Building dataset using Dataturks) I found a great plugin that enables batch image downloading on Google Chrome — this + Dataturks will make building training data a cakewalk. Linked . here You can try doing this with the image_classification tool of dataturks . Here the best feature which this tool provides is , if we have a unstructured dataset with all the images in a single folder. By manually labelling it with the classes , you can download a json file which has all the details of the image with the class embedded in it.Then use the scripts given there for and : here keras tensorflow > for tensorflow python3 tensorflow_json_parser.py — json_file “flower.json” — dataset_path “Dataset5/” > for keras python3 keras_json_parser.py 80 20 ------- ------- --json_file "flower.json" --dataset_path "Dataset5/" --train_percentage --validation_percentage Training Now it’s time to train the model. In tensorflow-for-poets-2 folder , there is folder called scripts which has everything required for re-training of a model. The has a special way of cropping and scaling the images which is too cool. retrain.py Then use the following command to train where the options name itself describe the required paths to train: python3 -m scripts.retrain \ =tf_files/bottlenecks \ =tf_files/models/inception \ =tf_files/retrained_graph.pb \ =tf_files/retrained_labels.txt \ =tf_files/flower_photos --bottleneck_dir --model_dir --output_graph --output_labels --image_dir This downloads the inception model and trains the last layer accordingly using the training folder and the arguments given. I trained it for 4000 steps on a GCP instance with 12GB Nvidia Tesla k80 and 7GB Vram. The training has been done with 80–20 , test- train split and we can see above , it gave a test_accuracy of 91% . Now its time to test! We have a .pb file in which can be used to test . The following changes have been added to the tf_files/ label_image.py PIL Image,ImageDraw,ImageFont results = results.tolist() image = Image.open(file_name) fonttype = ImageFont.truetype(“/usr/share/fonts/truetype/dejav/DejaVuSans.ttf”, ) draw = ImageDraw.Draw(image) draw.text(xy=( , ),text=str(labels[results.index(max(results))])+”:”+str(max(results)),fill = ( , , , ),font = fonttype) image.show() image.save(file_name.split(“.”)[ ]+” from import 18 5 5 255 255 255 128 0 1 "+”.jpg”) The above code will help us draw the accuracy on the image being tested and saves it .confidence percentages fora rodom image shown below Few outputs of testing are shown: (a collage of few outputs comprising of all classes) As we have seen the results were really promising enough for the task stated. KERAS: Keras is a high level API built on TensorFlow (and can be used on top of Theano too). It is more user-friendly and easy to use as compared to If we were a newbie to all this deep learning and wanted to write a new model from scratch, then Keras is what I would suggest for its ease in both readability and writability.It can be installed with: Tensorflow. pip keras install and even this thing is wrap over tensorflow, so again the CPU v/s GPU compatibility variations will apply here too. Since , we have to carry out the same task of classifying flowers using transfer learning with inception model , I’ve seen that Keras loads the model in a standard format like how the APIs are written. from keras import preprocess_input .applications .inception_v3 Keras has a standard format of loading the dataset i.e., instead of giving the folders directly within a dataset folder , we divide the train and test data manually and arrange them in the following manner. I have used the same dataset which I downloaded in the tensorflow section and made few changes as directed below. — Dataset folder - — train/ / — image — image / — image — image — test/ / — image — image / — image — image class 1 1 2 class 2 1 2 class 1 1 2 class 2 1 2 It should look like something below: and followed by that train and test should have folders as shown below: (TRAIN FOLDER) As, we are now done with the set up of the dataset , it’s time for training ! I have written down a small piece of code to do the training which goes below: os sys glob argparse matplotlib.pyplot plt keras __version__ keras.applications.inception_v3 InceptionV3, preprocess_input keras.models Model keras.layers Dense, GlobalAveragePooling2D keras.preprocessing.image ImageDataGenerator keras.optimizers SGD IM_WIDTH, IM_HEIGHT = , NB_EPOCHS = BAT_SIZE = FC_SIZE = NB_IV3_LAYERS_TO_FREEZE = os.path.exists(directory): cnt = r, dirs, files os.walk(directory): dr dirs: cnt += len(glob.glob(os.path.join(r, dr + ))) cnt layer base_model.layers: layer.trainable = model.compile(optimizer= , loss= , metrics=[ ]) x = base_model.output x = GlobalAveragePooling2D()(x) x = Dense(FC_SIZE, activation= )(x) predictions = Dense(nb_classes, activation= )(x) model = Model(input=base_model.input, output=predictions) model layer model.layers[:NB_IV3_LAYERS_TO_FREEZE]: layer.trainable = layer model.layers[NB_IV3_LAYERS_TO_FREEZE:]: layer.trainable = model.compile(optimizer=SGD(lr= , momentum= ), loss= , metrics=[ ]) nb_train_samples = get_nb_files(args.train_dir) nb_classes = len(glob.glob(args.train_dir + )) nb_val_samples = get_nb_files(args.val_dir) nb_epoch = int(args.nb_epoch) batch_size = int(args.batch_size) train_datagen = ImageDataGenerator( preprocessing_function=preprocess_input, rotation_range= , width_shift_range= , height_shift_range= , shear_range= , zoom_range= , horizontal_flip= ) test_datagen = ImageDataGenerator( preprocessing_function=preprocess_input, rotation_range= , width_shift_range= , height_shift_range= , shear_range= , zoom_range= , horizontal_flip= ) train_generator = train_datagen.flow_from_directory( args.train_dir, target_size=(IM_WIDTH, IM_HEIGHT), batch_size=batch_size, ) validation_generator = test_datagen.flow_from_directory( args.val_dir, target_size=(IM_WIDTH, IM_HEIGHT), batch_size=batch_size, ) base_model = InceptionV3(weights= , include_top= ) model = add_new_last_layer(base_model, nb_classes) setup_to_transfer_learn(model, base_model) history_tl = model.fit_generator( train_generator, nb_epoch=nb_epoch, samples_per_epoch=nb_train_samples, validation_data=validation_generator, nb_val_samples=nb_val_samples, class_weight= ) setup_to_finetune(model) history_ft = model.fit_generator( train_generator, samples_per_epoch=nb_train_samples, nb_epoch=nb_epoch, validation_data=validation_generator, nb_val_samples=nb_val_samples, class_weight= ) model.save(args.output_model_file) args.plot: plot_training(history_ft) acc = history.history[ ] val_acc = history.history[ ] loss = history.history[ ] val_loss = history.history[ ] epochs = range(len(acc)) plt.plot(epochs, acc, ) plt.plot(epochs, val_acc, ) plt.title( ) plt.figure() plt.plot(epochs, loss, ) plt.plot(epochs, val_loss, ) plt.title( ) plt.show() __name__== : a = argparse.ArgumentParser() a.add_argument( ) a.add_argument( ) a.add_argument( , default=NB_EPOCHS) a.add_argument( , default=BAT_SIZE) a.add_argument( , default= ) a.add_argument( , action= ) args = a.parse_args() args.train_dir args.val_dir : a.print_help() sys.exit( ) ( os.path.exists(args.train_dir)) ( os.path.exists(args.val_dir)): print( ) sys.exit( ) train(args) import import import import import as from import from import from import from import from import from import 299 299 #fixed size for InceptionV3 3 32 1024 172 : def get_nb_files (directory) """Get number of files by searching directory recursively""" if not return 0 0 for in for in "/*" return : def setup_to_transfer_learn (model, base_model) """Freeze all layers and compile the model""" for in False 'rmsprop' 'categorical_crossentropy' 'accuracy' : def add_new_last_layer (base_model, nb_classes) """Add last layer to the convnet Args: base_model: keras model excluding top nb_classes: # of classes Returns: new keras model with last layer """ 'relu' #new FC layer, random init 'softmax' #new softmax layer return : def setup_to_finetune (model) """Freeze the bottom NB_IV3_LAYERS and retrain the remaining top layers. note: NB_IV3_LAYERS corresponds to the top 2 inception blocks in the inceptionv3 arch Args: model: keras model """ for in False for in True 0.0001 0.9 'categorical_crossentropy' 'accuracy' : def train (args) """Use transfer learning and fine-tuning to train a network on a new dataset""" "/*" # data prep 30 0.2 0.2 0.2 0.2 True 30 0.2 0.2 0.2 0.2 True # setup model 'imagenet' False #include_top=False excludes final FC layer # transfer learning 'auto' # fine-tuning 'auto' if : def plot_training (history) 'acc' 'val_acc' 'loss' 'val_loss' 'r.' 'r' 'Training and validation accuracy' 'r.' 'r-' 'Training and validation loss' if "__main__" " - train_dir" " - val_dir" " - nb_epoch" " - batch_size" " - output_model_file" "inceptionv3-ft.model" " - plot" "store_true" if is None or is None 1 if not or not "directories do not exist" 1 and this code is neatly written and can be easily understood with the arguments being passed to the below command: python3 inception_train.py — train_dir flower_photos/train — val_dir flower_photos/validation — nb_epoch — batch_size — output_model_file inception_yo1.model \ \ 50 \ 10 \ and the training on my GPU took around 1 minute per epoch with 292 steps per epoch and was trained for 50 epochs (which is very much more ! ) with a batch size of 10 and a 80–20 data split. Whoop! we are done with training and achieved test_accuracy of ~91% and a loss of 0.38. The model has been saved as a inception.model file which can be loaded again and tested . To do that , another script has been written along with plotting the predicted class on the image and saving it. The testing script goes as below: sys argparse numpy np PIL Image requests io BytesIO matplotlib.pyplot plt PIL Image,ImageDraw,ImageFont keras.preprocessing image keras.models load_model keras.applications.inception_v3 preprocess_input target_size = ( , ) img.size != target_size: img = img.resize(target_size) x = image.img_to_array(img) x = np.expand_dims(x, axis= ) x = preprocess_input(x) preds = model.predict(x) preds[ ] plt.imshow(image) plt.axis( ) plt.figure() labels = ( , , , , ) plt.barh([ , , , , ], preds, alpha= ) plt.yticks([ , , , , ], labels) plt.xlabel( ) plt.xlim( , ) plt.tight_layout() plt.show() __name__== : a = argparse.ArgumentParser() a.add_argument( , help= ) a.add_argument( , help= ) a.add_argument( ) args = a.parse_args() args.image args.image_url : a.print_help() sys.exit( ) model = load_model(args.model) model.fit() args.image : labels = ( , , , , ) image1 = Image.open(args.image) preds = predict(model, image1, target_size) print(preds) preds = preds.tolist() plot_preds(image1, preds) fonttype = ImageFont.truetype( , ) draw = ImageDraw.Draw(image1) draw.text(xy=( , ),text = str(labels[preds.index(max(preds))])+ +str(max(preds)),fill = ( , , , ),font = fonttype) image1.show() image1.save((args.image).split( )[ ]+ + ) import import import as from import import from import import as from import from import from import from import 229 229 #fixed size for InceptionV3 architecture : def predict (model, img, target_size) """Run model prediction on image Args: model: keras model img: PIL format image target_size: (w,h) tuple Returns: list of predicted labels and their probabilities """ if 0 return 0 : def plot_preds (image, preds) """Displays image and the top-n predicted probabilities in a bar graph Args: image: PIL image preds: list of predicted labels and their probabilities """ 'off' "daisy" "dandelion" "roses" "sunflower" "tulips" 0 1 2 3 4 0.5 0 1 2 3 4 'Probability' 0 1.01 if "__main__" " - image" "path to image" " - image_url" "url to image" " - model" if is None and is None 1 if is not None "daisy" "dandelion" "roses" "sunflower" "tulips" "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf" 18 5 5 ":" 255 255 255 128 "." 0 "1" ".jpg" (inception_test.py) This script can be tested as: python3 -m scripts.label_image — =tf_files/retrained_graph.pb — =rose.jpeg graph image The predicted confidence percentages over all classes is outputted like: [daisy,dandelion,roses,sunflower,tulip] and below are the few outputs with graph: (tested images with their probability graphs) Finally! you have learnt how to build a powerful classifier using both Keras and tensorflow. But , which one is best is still a question in our heads! So , let us do a comparative study only based on this classification task as of now. The whole tain and test code of keras along with the changed scripts of tensorflow are available in my github . here Prototyping: If you really want to write a code quickly and build a model , then Keras is a go. We can build complex models within minutes! The and the APIs are so powerful that they wont even give you a sense that you are the building powerful models due to the ease in using them. Model Sequential model = Sequential() model.add(Dense( , activation= , input_dim= )) model.add(Dense( , activation= )) model.compile(optimizer= , loss= , metrics=[ ]) numpy np data = np.random.random(( , )) labels = np.random.randint( , size=( , )) model.fit(data, labels, epochs= , batch_size= ) 32 'relu' 100 1 'sigmoid' 'rmsprop' 'binary_crossentropy' 'accuracy' # Generate dummy data import as 1000 100 2 1000 1 # Train the model, iterating on the data in batches of 32 samples 10 32 That’s it, a model is ready! Even transfer learning is easy to code in Keras than in tensorflow. Tensorflow is too tough to code from scratch until you are a sticky coder. Scratch Coding and flexibility: As tensorflow is a low-level library when compared to Keras , many new functions can be implemented in a better way in tensorflow than in Keras for example , any activation fucntion etc… And also the fine-tuning and tweaking of the model is very flexible in tensorflow than in Keras due to much more parameters being available. Training time and processing power: The above models were trained on the same dataset , we see that Keras takes loner time to train than tensorflow . Tensorflow finished the training of 4000 steps in 15 minutes where as Keras took around 2 hours for 50 epochs . May be we cannot compare steps with epochs , but of you see in this case , both gave a test accuracy of 91% which is comparable and we can depict that keras trains a bit slower than tensorflow. Apart from this , it makes sense because of tensorflow being a low level library. Extra features provided: Tensorflow has a inbuilt debugger which can debug during the training as well as generating the graphs. (TensorFlow Debugger snapshot. Source: TensorFlow documentation) Tensorflow even supports threads and queues to train the heavy tensors asynchronously! This provides TPUs a better and much faster processing speeds.Sample code for threads is shown below: init_op = tf.global_variables_initializer() sess = tf.Session() sess.run(init_op) coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(sess=sess, coord=coord) : coord.should_stop(): sess.run(train_op) tf.errors.OutOfRangeError: print( ) : coord.request_stop() coord.join(threads) sess.close() # Create the graph, etc. # Create a session for running operations in the Graph. # Initialize the variables (like the epoch counter). # Start input enqueue threads. try while not # Run training steps or whatever except 'Done training -- epoch limit reached' finally # When done, ask the threads to stop. # Wait for threads to finish. Monitoring and Control: According to my experience in dee learning , I feel tensorflow is highly suitable for many cases though it is little tough. For example , we can monitor each and everything very easily such as controlling the weights , gradients of your network. We can choose which step should be trained and which should not . This is not that feasible in Keras.The below given line does that magic! = tf.Variable(1, = , =tf.int32) step trainable False dtype Conclusion Anyways, Keras is going to be integrated in tensorflow shortly! So, why go pythonic?(Keras is pythonic ) . Spend some time and get used to tensorflow is what I suggest . The classification problem above , if you have followed the blog and done the steps accordingly , then you will feel that Keras is little painful and patience killer than tensorflow in many aspects. So , try using other classes and try training classifers for applications like fake note detection etc… Hope this blog would have given you a better insight to what to use when ! I would love to hear any suggestions or queries. Please write to me at sameer.gadicherla@dataturks.com