Too Long; Didn't Read
Pretrained Artificial Neural Networks used to work like a Blackbox: You hand them an input and they predict an output with a certain probability — but without us knowing the internal processes of how they came up with their prediction. A Neural Network to recognize images usually consists of around 20 neuron layers, trained with millions of images to tweak the network parameters to give high quality classifications. With the basic knowledge of what individual layers are roughly doing, a technique is needed to gather insights about an already trained network.