Approach Pre-Trained Deep Learning Models With Caution

Too Long; Didn't Read

Every major framework offers pre-trained models like Inception V3, ResNet, AlexNet with weights: With almost all torchvision models, they all use the same model as the corresponding model-level module for the corresponding training function. Some models using batch normalization can be unreliable, and some models using forward-pass evaluations (with gradients supposedly off) still result in weights changing at inference time. The benchmarks on Keras Applications cannot be reproduced, even when exactly copying the example code. In fact, their reported accuracies are usually higher than the actual accuracy of the models.