Maven dependency exclusion not working

This article helps you in such cases: There are at list two reasons of such problems: 1. This dependency is transitive for other dependency, not that one, where you made exclusion. Thus you need to add exclusion tag somewhere else (if your exclusion is located in section, not plugin-level). mvn dependency:tree is very useful for […]

READ MORE

log4j 1.x to log4j 2.x bridge migration

Even though you may want to merely copy & paste the code below, I suggest you firstly explore carefully the article describing how Java logging works, what is slf4j, what is log4j, what is bridge and how it works: https://mchesnavsky.tech/how-the-java-logging-should-works/, and then return back. In this article you’ll find the answers for these questions: Suppose […]

READ MORE

How the Java logging should works

In this article you will find the answers for these questions: Let’s start To begin with, there is two main concepts in Java logging: The problem Suppose the you’re Java library developer (let’s say it’s called our_library), not standard Java application developer. You need to have logging in your library. Suppose you have chosen log4j. […]

READ MORE

Spark Java access remote HDFS

Suppose we need to work with different HDFS (clusterB, for instance) from our Spark Java application, running on clusterA. Firstly, you need to add –conf key to your run command. Depends on Spark version: Secondly, when you creating Spark’s Java context, add that: You need to go to clusterB and gather core-site.xml and hdfs-site.xml from there (default location for Cloudera is /etc/hadoop/conf) […]

READ MORE

Java access remote HDFS from current Hadoop cluster

Suppose we have our Java app running on Hadoop clusterA, and we want to access remote HDFS based on Hadoop clusterB. Let’s see how we can do it: You need to go to clusterB and gather core-site.xml and hdfs-site.xml from there (default location for Cloudera is /etc/hadoop/conf) and put near your app running in clusterA. […]

READ MORE

Java SSL certificate revocation check

There is two common way to check TLS certificate revocation status: Certificate Revocation List (CRL) Online Certificate Status Protocol (OCSP) The second option is more faster and modern way to do that. The OCSP link must be presented some way to do that. There are at least two options: Your Certificate Authority (CA) automatically puts […]

READ MORE

Kafka client gets stuck in CHECKING API VERSIONS state

If you face the problems, when: Kafka Consumer is not involving in rebalance process Kafka Consumer is not reading messages Partitions are not allocated to the Kafka Consumer, but the Consumer is subscribed to the topic Kafka Consumer connection gets stuck in CHECKING_API_VERSIONS state Kafka Producer connection gets stuck in CHECKING_API_VERSIONS state Then this is […]

READ MORE