simply about difficult
In this article you will find the answers for these questions: Let’s start To begin with, there is two main concepts in Java logging: The problem Suppose the you’re Java library developer (let’s say it’s called our_library), not standard Java application developer. You need to have logging in your library. Suppose you have chosen log4j. […]
READ MOREOne day you may face this error: Full exception text is: Firstly, you need to check that user from which your Spark working on, has home folder under /user like that: /user/John_Doe. If you already have that folder, try to grant access to /user dir. If you unable to grant access to that directory or […]
READ MORESuppose we need to work with different HDFS (clusterB, for instance) from our Spark Java application, running on clusterA. Firstly, you need to add –conf key to your run command. Depends on Spark version: Secondly, when you creating Spark’s Java context, add that: You need to go to clusterB and gather core-site.xml and hdfs-site.xml from there (default location for Cloudera is /etc/hadoop/conf) […]
READ MORESuppose we have our Java app running on Hadoop clusterA, and we want to access remote HDFS based on Hadoop clusterB. Let’s see how we can do it: You need to go to clusterB and gather core-site.xml and hdfs-site.xml from there (default location for Cloudera is /etc/hadoop/conf) and put near your app running in clusterA. […]
READ MOREThere is two common way to check TLS certificate revocation status: Certificate Revocation List (CRL) Online Certificate Status Protocol (OCSP) The second option is more faster and modern way to do that. The OCSP link must be presented some way to do that. There are at least two options: Your Certificate Authority (CA) automatically puts […]
READ MOREIf you want to describe Kafka Consumer group in Java, you may want to use this method:
READ MOREIf you want to gather information about the interaction of Kafka Consumer with Kafka Broker, you may want to use this method: Helper methods:
READ MOREIf you want to read clientid from Kafka Consumer, you may use this method:
READ MORESuppose that you need to create Kafka Java consumer with Kerberos. The code will be: You don’t need to specify java.security.auth.login.config Java property, because we set SaslConfigs.SASL_JAAS_CONFIG property directly to the consumer. You just need to made changes in kafkaJaasConfiguration() method that necessary for your Kerberos configuration.
READ MOREThe problem can be appears in Hadoop’s NodeManager logs. Usually it means that NodeManager is trying to use an expired / not renewed HDFS delegation token. For example, you can face this error while app log aggregation process. The timeline is: Your application pass HDFS delegation token to the NodeManager through the ContainerLaunchContext class, because […]
READ MORE