Background on Autoencoder and SPRT and Autoencoder

Table of Links

Abstract and 1. Introduction

2. Related Work

3. Background on Autoencoder and SPRT and 3.1. Autoencoder

3.2. Sequential Probability Ratio Test

4. Design of CUMAD and 4.1. Network Model

4.2. CUMAD: Cumulative Anomaly Detection

5. Evaluation Studies and 5.1. Dataset, Features, and CUMAD System Setup

5.2. Performance Results

6. Conclusions and References

3. Background on Autoencoder and SPRT

In this section we provide the necessary background on autoencoder and sequential probability ratio test (SPRT) for understanding the development of the proposed CUMAD framework. We refer interested readers to [6] and [7], respectively, for the detailed treatment on these two topics.

3.1. Autoencoder

Autoencoder is an unsupervised neutral network that aims to reconstruct the input at the output. Figure 1 illustrates a simple standard (undercomplete) autoencoder.

An autoencoder can be considered as consisting of two components: an encoder f and an decoder g. Given input data x, the encoder function f maps x to a latent-space representation, or code h, that is h = f(x). Using the corresponding code h as the input, the decoder function g tries to reconstruct the original input x at its output x ′, that is, x′ = g(h). Combining both the encoder function and decoder function together, we have x′ = g(f(x)). Let L(x, x′) be the reconstruction error, that is, the difference between x and x′. The autoenceder aims to minimize L(x, x ′). We note that there are different definitions of L(x, x′) and one of the most common definitions is the mean squared errors (MSE). We note that in the example autoencoder of Figure 1, both the encoder and decoder have only one hidden layer. This is only for illustration purpose. In reality they can have many hidden layers, depending on the specific application requirement.

Autoencoders have been traditionally used in applications of dimensionality reduction and feature learning, by focusing on the compressed code of an autoencoder, which holds the latent-space representation of the original data. On the other hand, autoencoders also possess a few desired properties, making them an attractive candidate for anomaly detection. For example, an autoencoder is able to extract the salient features of the original data to remove dependency in the original data. More importantly, an autoencoder can only learn the properties or distributions of the data that it has seen during the training stage, that is, the data points in the training dataset. It excels at reconstructing data that are similar to the training data, but performs poorly on data that are very different from the training data, in terms of the reconstruction error L(x, x′).

This is an appealing property of autoencoders in the application of anomaly detection. For example, in the context of detecting compromised IoT devices, we can establish the normal behavioral model of an IoT device using an autoencoder by training it with benign network traffic before the device has been compromised. We can continue monitoring the IoT device by passing the corresponding network traffic of the device into the trained model. If the reconstruction error is no greater than a pre-specified threshold, we consider the corresponding network traffic to be benign. When the reconstruction error is greater than the threshold, we claim that the network traffic is anomalous.