A Difference Between DL and Statistics

One thing that I love about being in grad school is the unending innovation that reverberates in the corridors. Sure, I sit in a cubicle part-time coding my life away, but there moments where people step out of their hole to converse with those around them. are One of these structured ways is through the weekly update meetings, and one of the many conversations we have inspired this post. Let’s start off with statistics. One of statistics’ main focus is to create a generalizable model, such as a linear or multivariate regression model, to best fit the data to represent the pattern you are investigating. There are other key topics in statistics like statistical significance, , probability theory, and model evaluation that are shared with the community. However, we like abstraction, so let’s stick with regression for now. correlation vs. causation machine learning Linear regression is powerful and easy to understand. However, the errors that are associated with each data point get accumulated such that the inherent nature of the point is lost with the goal of minimizing the global loss/error. This is a clear trade-off. Now, don’t get me wrong, systems still try to minimize the loss throughout the system, and there are many clever ways to complete these optimizations. deep learning However, one thing that the deep learning approach versus the approach above (though there are many) is now we can perturb the model to understand what drove the algorithm to influence its decision. differentiates We can see the importance of features on the through point by point investigation, rather than relying on statistics to generalize on the global level. individual level We can now peer into the complex black box, and people are definitely starting to. Photo by on Dhruv Deshmukh Unsplash There are now more coming out to dissect the activations or neurons that drove the model’s decision. These are visually useful in imaging applications because we can actually see what parts helped the model. methods Further, these methods can be used in a better understanding of the model’s performance — i.e. where the model predicted correctly, where the model messed up, where the model was learning from noise. Or maybe most significantly, where the model can drive by uncovering important features not previously thought of as important. new discoveries Maybe deep learning can be a tool for further understanding a problem like neurodegeneration or time-sensitive areas of concern to a predictive tool. in addition can lead to the next frontier or scientific discovery. Activation exploration Thanks for reading.