Content Overview Keras preprocessing
Available preprocessing
Text preprocessing
Numerical features preprocessing
Categorical features preprocessing
Image preprocessing
Image data augmentation
The adapt() method
Preprocessing data before the model or inside the model
Benefits of doing preprocessing inside the model at inference time
Preprocessing during multi-worker training Keras preprocessing Available preprocessing Text preprocessing Numerical features preprocessing Categorical features preprocessing Image preprocessing Image data augmentation The adapt() method Preprocessing data before the model or inside the model Benefits of doing preprocessing inside the model at inference time Preprocessing during multi-worker training Keras preprocessing The Keras preprocessing layers API allows developers to build Keras-native input processing pipelines. These input processing pipelines can be used as independent preprocessing code in non-Keras workflows, combined directly with Keras models, and exported as part of a Keras SavedModel. With Keras preprocessing layers, you can build and export models that are truly end-to-end: models that accept raw images or raw structured data as input; models that handle feature normalization or feature value indexing on their own. Available preprocessing Text preprocessing tf.keras.layers.TextVectorization: turns raw strings into an encoded representation that can be read by an Embedding layer or Dense layer. tf.keras.layers.TextVectorization: turns raw strings into an encoded representation that can be read by an Embedding layer or Dense layer. tf.keras.layers.TextVectorization Embedding Dense Numerical features preprocessing tf.keras.layers.Normalization: performs feature-wise normalization of input features.
tf.keras.layers.Discretization: turns continuous numerical features into integer categorical features. tf.keras.layers.Normalization: performs feature-wise normalization of input features. tf.keras.layers.Normalization tf.keras.layers.Discretization: turns continuous numerical features into integer categorical features. tf.keras.layers.Discretization Categorical features preprocessing tf.keras.layers.CategoryEncoding: turns integer categorical features into one-hot, multi-hot, or count dense representations.
tf.keras.layers.Hashing: performs categorical feature hashing, also known as the "hashing trick".
tf.keras.layers.StringLookup: turns string categorical values into an encoded representation that can be read by an Embedding layer or Dense layer.
tf.keras.layers.IntegerLookup: turns integer categorical values into an encoded representation that can be read by an Embedding layer or Dense layer. tf.keras.layers.CategoryEncoding: turns integer categorical features into one-hot, multi-hot, or count dense representations. tf.keras.layers.CategoryEncoding tf.keras.layers.Hashing: performs categorical feature hashing, also known as the "hashing trick". tf.keras.layers.Hashing tf.keras.layers.StringLookup: turns string categorical values into an encoded representation that can be read by an Embedding layer or Dense layer. tf.keras.layers.StringLookup Embedding Dense tf.keras.layers.IntegerLookup: turns integer categorical values into an encoded representation that can be read by an Embedding layer or Dense layer. tf.keras.layers.IntegerLookup Embedding Dense Image preprocessing These layers are for standardizing the inputs of an image model. tf.keras.layers.Resizing: resizes a batch of images to a target size.
tf.keras.layers.Rescaling: rescales and offsets the values of a batch of images (e.g. go from inputs in the [0, 255] range to inputs in the [0, 1] range.
tf.keras.layers.CenterCrop: returns a center crop of a batch of images. tf.keras.layers.Resizing: resizes a batch of images to a target size. tf.keras.layers.Resizing tf.keras.layers.Rescaling: rescales and offsets the values of a batch of images (e.g. go from inputs in the [0, 255] range to inputs in the [0, 1] range. tf.keras.layers.Rescaling [0, 255] [0, 1] tf.keras.layers.CenterCrop: returns a center crop of a batch of images. tf.keras.layers.CenterCrop Image data augmentation These layers apply random augmentation transforms to a batch of images. They are only active during training. tf.keras.layers.RandomCrop
tf.keras.layers.RandomFlip
tf.keras.layers.RandomTranslation
tf.keras.layers.RandomRotation
tf.keras.layers.RandomZoom
tf.keras.layers.RandomContrast tf.keras.layers.RandomCrop tf.keras.layers.RandomCrop tf.keras.layers.RandomFlip tf.keras.layers.RandomFlip tf.keras.layers.RandomTranslation tf.keras.layers.RandomTranslation tf.keras.layers.RandomRotation tf.keras.layers.RandomRotation tf.keras.layers.RandomZoom tf.keras.layers.RandomZoom tf.keras.layers.RandomContrast tf.keras.layers.RandomContrast The adapt() method adapt() Some preprocessing layers have an internal state that can be computed based on a sample of the training data. The list of stateful preprocessing layers is: TextVectorization: holds a mapping between string tokens and integer indices
StringLookup and IntegerLookup: hold a mapping between input values and integer indices.
Normalization: holds the mean and standard deviation of the features.
Discretization: holds information about value bucket boundaries. TextVectorization: holds a mapping between string tokens and integer indices TextVectorization StringLookup and IntegerLookup: hold a mapping between input values and integer indices. StringLookup IntegerLookup Normalization: holds the mean and standard deviation of the features. Normalization Discretization: holds information about value bucket boundaries. Discretization Crucially, these layers are non-trainable. Their state is not set during training; it must be set before training, either by initializing them from a precomputed constant, or by "adapting" them on data. non-trainable before training You set the state of a preprocessing layer by exposing it to training data, via the adapt() method: adapt() import numpy as np
import tensorflow as tf
from tensorflow import keras
from keras import layers

data = np.array(
    [
        [0.1, 0.2, 0.3],
        [0.8, 0.9, 1.0],
        [1.5, 1.6, 1.7],
    ]
)
layer = layers.Normalization()
layer.adapt(data)
normalized_data = layer(data)

print("Features mean: %.2f" % (normalized_data.numpy().mean()))
print("Features std: %.2f" % (normalized_data.numpy().std())) import numpy as np
import tensorflow as tf
from tensorflow import keras
from keras import layers

data = np.array(
    [
        [0.1, 0.2, 0.3],
        [0.8, 0.9, 1.0],
        [1.5, 1.6, 1.7],
    ]
)
layer = layers.Normalization()
layer.adapt(data)
normalized_data = layer(data)

print("Features mean: %.2f" % (normalized_data.numpy().mean()))
print("Features std: %.2f" % (normalized_data.numpy().std())) Features mean: -0.00
Features std: 1.00 Features mean: -0.00
Features std: 1.00 The adapt() method takes either a Numpy array or a tf.data.Dataset object. In the case of StringLookup and TextVectorization, you can also pass a list of strings: adapt() tf.data.Dataset StringLookup TextVectorization data = [
    "ξεῖν᾽, ἦ τοι μὲν ὄνειροι ἀμήχανοι ἀκριτόμυθοι",
    "γίγνοντ᾽, οὐδέ τι πάντα τελείεται ἀνθρώποισι.",
    "δοιαὶ γάρ τε πύλαι ἀμενηνῶν εἰσὶν ὀνείρων:",
    "αἱ μὲν γὰρ κεράεσσι τετεύχαται, αἱ δ᾽ ἐλέφαντι:",
    "τῶν οἳ μέν κ᾽ ἔλθωσι διὰ πριστοῦ ἐλέφαντος,",
    "οἵ ῥ᾽ ἐλεφαίρονται, ἔπε᾽ ἀκράαντα φέροντες:",
    "οἱ δὲ διὰ ξεστῶν κεράων ἔλθωσι θύραζε,",
    "οἵ ῥ᾽ ἔτυμα κραίνουσι, βροτῶν ὅτε κέν τις ἴδηται.",
]
layer = layers.TextVectorization()
layer.adapt(data)
vectorized_text = layer(data)
print(vectorized_text) data = [
    "ξεῖν᾽, ἦ τοι μὲν ὄνειροι ἀμήχανοι ἀκριτόμυθοι",
    "γίγνοντ᾽, οὐδέ τι πάντα τελείεται ἀνθρώποισι.",
    "δοιαὶ γάρ τε πύλαι ἀμενηνῶν εἰσὶν ὀνείρων:",
    "αἱ μὲν γὰρ κεράεσσι τετεύχαται, αἱ δ᾽ ἐλέφαντι:",
    "τῶν οἳ μέν κ᾽ ἔλθωσι διὰ πριστοῦ ἐλέφαντος,",
    "οἵ ῥ᾽ ἐλεφαίρονται, ἔπε᾽ ἀκράαντα φέροντες:",
    "οἱ δὲ διὰ ξεστῶν κεράων ἔλθωσι θύραζε,",
    "οἵ ῥ᾽ ἔτυμα κραίνουσι, βροτῶν ὅτε κέν τις ἴδηται.",
]
layer = layers.TextVectorization()
layer.adapt(data)
vectorized_text = layer(data)
print(vectorized_text) tf.Tensor(
[[37 12 25  5  9 20 21  0  0]
 [51 34 27 33 29 18  0  0  0]
 [49 52 30 31 19 46 10  0  0]
 [ 7  5 50 43 28  7 47 17  0]
 [24 35 39 40  3  6 32 16  0]
 [ 4  2 15 14 22 23  0  0  0]
 [36 48  6 38 42  3 45  0  0]
 [ 4  2 13 41 53  8 44 26 11]], shape=(8, 9), dtype=int64) tf.Tensor(
[[37 12 25  5  9 20 21  0  0]
 [51 34 27 33 29 18  0  0  0]
 [49 52 30 31 19 46 10  0  0]
 [ 7  5 50 43 28  7 47 17  0]
 [24 35 39 40  3  6 32 16  0]
 [ 4  2 15 14 22 23  0  0  0]
 [36 48  6 38 42  3 45  0  0]
 [ 4  2 13 41 53  8 44 26 11]], shape=(8, 9), dtype=int64) In addition, adaptable layers always expose an option to directly set state via constructor arguments or weight assignment. If the intended state values are known at layer construction time, or are calculated outside of the adapt() call, they can be set without relying on the layer's internal computation. For instance, if external vocabulary files for the TextVectorization, StringLookup, or IntegerLookup layers already exist, those can be loaded directly into the lookup tables by passing a path to the vocabulary file in the layer's constructor arguments. adapt() TextVectorization StringLookup IntegerLookup Here's an example where you instantiate a StringLookup layer with precomputed vocabulary: StringLookup vocab = ["a", "b", "c", "d"]
data = tf.constant([["a", "c", "d"], ["d", "z", "b"]])
layer = layers.StringLookup(vocabulary=vocab)
vectorized_data = layer(data)
print(vectorized_data) vocab = ["a", "b", "c", "d"]
data = tf.constant([["a", "c", "d"], ["d", "z", "b"]])
layer = layers.StringLookup(vocabulary=vocab)
vectorized_data = layer(data)
print(vectorized_data) tf.Tensor(
[[1 3 4]
 [4 0 2]], shape=(2, 3), dtype=int64) tf.Tensor(
[[1 3 4]
 [4 0 2]], shape=(2, 3), dtype=int64) Preprocessing data before the model or inside the model There are two ways you could be using preprocessing layers: Option 1: Make them part of the model, like this: Option 1: inputs = keras.Input(shape=input_shape)
x = preprocessing_layer(inputs)
outputs = rest_of_the_model(x)
model = keras.Model(inputs, outputs) inputs = keras.Input(shape=input_shape)
x = preprocessing_layer(inputs)
outputs = rest_of_the_model(x)
model = keras.Model(inputs, outputs) With this option, preprocessing will happen on device, synchronously with the rest of the model execution, meaning that it will benefit from GPU acceleration. If you're training on a GPU, this is the best option for the Normalization layer, and for all image preprocessing and data augmentation layers. Normalization Option 2: apply it to your tf.data.Dataset, so as to obtain a dataset that yields batches of preprocessed data, like this: Option 2: tf.data.Dataset dataset = dataset.map(lambda x, y: (preprocessing_layer(x), y)) dataset = dataset.map(lambda x, y: (preprocessing_layer(x), y)) With this option, your preprocessing will happen on a CPU, asynchronously, and will be buffered before going into the model. In addition, if you call dataset.prefetch(tf.data.AUTOTUNE) on your dataset, the preprocessing will happen efficiently in parallel with training: dataset.prefetch(tf.data.AUTOTUNE) dataset = dataset.map(lambda x, y: (preprocessing_layer(x), y))
dataset = dataset.prefetch(tf.data.AUTOTUNE)
model.fit(dataset, ...) dataset = dataset.map(lambda x, y: (preprocessing_layer(x), y))
dataset = dataset.prefetch(tf.data.AUTOTUNE)
model.fit(dataset, ...) This is the best option for TextVectorization, and all structured data preprocessing layers. It can also be a good option if you're training on a CPU and you use image preprocessing layers. TextVectorization Note that the TextVectorization layer can only be executed on a CPU, as it is mostly a dictionary lookup operation. Therefore, if you are training your model on a GPU or a TPU, you should put the TextVectorization layer in the tf.data pipeline to get the best performance. TextVectorization TextVectorization tf.data When running on a TPU, you should always place preprocessing layers in the tf.data pipeline (with the exception of Normalization and Rescaling, which run fine on a TPU and are commonly used as the first layer in an image model). When running on a TPU, you should always place preprocessing layers in the tf.data Normalization Rescaling Benefits of doing preprocessing inside the model at inference time Even if you go with option 2, you may later want to export an inference-only end-to-end model that will include the preprocessing layers. The key benefit to doing this is that it makes your model portable and it helps reduce the training/serving skew. it makes your model portable helps reduce the training/serving skew training/serving skew When all data preprocessing is part of the model, other people can load and use your model without having to be aware of how each feature is expected to be encoded & normalized. Your inference model will be able to process raw images or raw structured data, and will not require users of the model to be aware of the details of e.g. the tokenization scheme used for text, the indexing scheme used for categorical features, whether image pixel values are normalized to [-1, +1] or to [0, 1], etc. This is especially powerful if you're exporting your model to another runtime, such as TensorFlow.js: you won't have to reimplement your preprocessing pipeline in JavaScript. [-1, +1] [0, 1] If you initially put your preprocessing layers in your tf.data pipeline, you can export an inference model that packages the preprocessing. Simply instantiate a new model that chains your preprocessing layers and your training model: tf.data inputs = keras.Input(shape=input_shape)
x = preprocessing_layer(inputs)
outputs = training_model(x)
inference_model = keras.Model(inputs, outputs) inputs = keras.Input(shape=input_shape)
x = preprocessing_layer(inputs)
outputs = training_model(x)
inference_model = keras.Model(inputs, outputs) Preprocessing during multi-worker training Preprocessing layers are compatible with the tf.distribute API for running training across multiple machines. tf.distribute In general, preprocessing layers should be placed inside a tf.distribute.Strategy.scope() and called either inside or before the model as discussed above. tf.distribute.Strategy.scope() with strategy.scope():
    inputs = keras.Input(shape=input_shape)
    preprocessing_layer = tf.keras.layers.Hashing(10)
    dense_layer = tf.keras.layers.Dense(16) with strategy.scope():
    inputs = keras.Input(shape=input_shape)
    preprocessing_layer = tf.keras.layers.Hashing(10)
    dense_layer = tf.keras.layers.Dense(16) For more details, refer to the Data preprocessing section of the Distributed input tutorial. Data preprocessing Distributed input Originally published on the TensorFlow website, this article appears here under a new headline and is licensed under CC BY 4.0. Code samples shared under the Apache 2.0 License. Originally published on the TensorFlow website, this article appears here under a new headline and is licensed under CC BY 4.0. Code samples shared under the Apache 2.0 License. Originally published on the TensorFlow website, this article appears here under a new headline and is licensed under CC BY 4.0. Code samples shared under the Apache 2.0 License. TensorFlow TensorFlow

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Apache

How to Build RNNs in Keras

TensorBoard, Checkpoints, and Custom Hooks in Keras

Too Long; Didn't Read

Make resilience your competitive advantage

Beginner’s Guide to Keras Preprocessing Layers

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Only TensorFlow Preprocessing Guide You Need

The Only TensorFlow Preprocessing Guide You Need

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps