Too Long; Didn't Read
Logging in Apache Spark comes very easy since Spark offers access to a <em>log </em>object out of the box. Only some configuration setups need to be done. In a <a href="https://medium.com/@anicolaspp/how-to-log-in-apache-spark-f4204fad78a#.7asl0yplw" target="_blank"><strong><em>previous post</em></strong></a><strong><em> </em></strong>we have looked at how to do this while showing some problems that may arise. However, the solution presented might cause some problems at the moment we want to collect the logs since they are distributed across the entire cluster. Even if we utilize <strong>Yarn</strong> log aggregation capabilities, there will be some contentions that might affect performance or even worse, in some cases we could end with log interleaves corrupting the nature of logs itself, they time ordered properties they should present.