Kafka broker Kerberos

Let’s see how we can configure Kerberos between Kafka broker and Kafka client on server side. The client side is presented here: https://mchesnavsky.tech/how-to-create-kafka-kerberos-java-consumer. <kafka_home>/conf/server.properties <kafka_home>/bin/kafka-run-class.sh Insert this: To KAFKA_OPTS: Result: /your/path/to/kafka_server_jaas.conf Kerberos between Kafka brokers is configuring with separate conf keys (which we not mentioned in this article). Above configuration is for broker-client interaction.

READ MORE

Yarn is not aggregating application logs

First of all, you need to check NodeManager logs. There may be at least two problems: Log aggregation is not initialized problem: https://mchesnavsky.tech/log-aggregation-is-not-initialized HDFS DELEGATION TOKEN can’t be found in cache problem: https://mchesnavsky.tech/hdfs-delegation-token-cant-be-found-in-cache Please, refer to corresponding article, or made a note in comments below, if you have any other problem.

READ MORE

Log aggregation is not initialized

You may encounter with Hadoop Yarn exception in NodeManager logs that states: It may happen because NM reboot. The newly launched NM inherited the running application, and it does not know how to collect logs from it.According to the Hadoop Yarn NodeManager source code, instances of log collector classes for each running application are stored […]

READ MORE

How to create Kafka Kerberos Java consumer

Suppose that you need to create Kafka Java consumer with Kerberos. The code will be: You don’t need to specify java.security.auth.login.config Java property, because we set SaslConfigs.SASL_JAAS_CONFIG property directly to the consumer. You just need to made changes in kafkaJaasConfiguration() method that necessary for your Kerberos configuration.

READ MORE

HDFS DELEGATION TOKEN can’t be found in cache

The problem can be appears in Hadoop’s NodeManager logs. Usually it means that NodeManager is trying to use an expired / not renewed HDFS delegation token. For example, you can face this error while app log aggregation process. The timeline is: Your application pass HDFS delegation token to the NodeManager through the ContainerLaunchContext class, because […]

READ MORE

Gmavenplus: Unrecognized target bytecode

When you try to compile Groovy with Java and use gmavenplus plugin for that purpose, you may face with this errors: To solve this problem, you need replace the ‘8’ to ‘1.8’ in your maven-compiler-plugin: If you don’t have maven-compiler-plugin – add it to your project’s pom.xml, as above.

READ MORE

Resource changed on src filesystem

Full exception text: This can happen when some process overwrites application files in HDFS application directory while app is running. An example of the situation: You start app instance_1, which stores the distribution files in the hdfs://tmp/app folder. After a while you start the second instance_2 which stores the distribution files in the same HDFS […]

READ MORE