Log aggregation is not initialized

You may encounter with Hadoop Yarn exception in NodeManager logs that states:

Log aggregation is not initialized for application_1657793849922_0033, did it fail to start?

It may happen because NM reboot. The newly launched NM inherited the running application, and it does not know how to collect logs from it.
According to the Hadoop Yarn NodeManager source code, instances of log collector classes for each running application are stored in ConcurrentHashMap.

When the application starts, NM puts log collector class object to HashMap. It is made once, and during the NM reboot, this cache is simply erased.

As a result, such an error is happens (according to the code it is issued if the required log collector instance is not exists in the map).

If you still have any questions, feel free to ask me in the comments under this article, or write me on promark33@gmail.com.

Leave a Reply

Your email address will not be published.