How to create Kafka Kerberos Java consumer

Suppose that you need to create Kafka Java consumer with Kerberos. The code will be: You don’t need to specify java.security.auth.login.config Java property, because we set SaslConfigs.SASL_JAAS_CONFIG property directly to the consumer. You just need to made changes in kafkaJaasConfiguration() method that necessary for your Kerberos configuration.

READ MORE

HDFS DELEGATION TOKEN can’t be found in cache

The problem can be appears in Hadoop’s NodeManager logs. Usually it means that NodeManager is trying to use an expired / not renewed HDFS delegation token. For example, you can face this error while app log aggregation process. The timeline is: Your application pass HDFS delegation token to the NodeManager through the ContainerLaunchContext class, because […]

READ MORE

Gmavenplus: Unrecognized target bytecode

When you try to compile Groovy with Java and use gmavenplus plugin for that purpose, you may face with this errors: To solve this problem, you need replace the ‘8’ to ‘1.8’ in your maven-compiler-plugin: If you don’t have maven-compiler-plugin – add it to your project’s pom.xml, as above.

READ MORE

Resource changed on src filesystem

Full exception text: This can happen when some process overwrites application files in HDFS application directory while app is running. An example of the situation: You start app instance_1, which stores the distribution files in the hdfs://tmp/app folder. After a while you start the second instance_2 which stores the distribution files in the same HDFS […]

READ MORE

How to generate heavy CPU load

Sometimes you need to generate heavy CPU load in your app for testing purposes. if you are using Spring, it will be very convenient to run multi-threaded password hashing: That code is non-blocking, so your program are not going to pause. If you want to pause, you may add the line to the end of […]

READ MORE