Send JSON as parameter in url using RestTemplate

If you try to send the json as a parameter in url via Spring’s RestTemplate, you will encounter the following error: In order to solve this problem, you need to: Encode the URL yourself Disable url encoding at the RestTemplate level. Let’s do that: Many thanks to Chris Liu for showing how to disable url […]

READ MORE

How to delete PushGateway groups

In this article, we are going to figure out, how to delete PushGateway metrics/groups. Let’s also answer the question – why “PushGateway metrics delete not working” or “PushGateway groups delete not working”. First of all, you must specify all the keys of the group that you want to delete.You need to delete groups with such […]

READ MORE

How to add URL cert to JKS

Let’s imagine that you need to download a certificate from a website / service / endpoint and put it in the trusted certificates store (JKS or cacerts). To do this, you need to fill in the parameters SERVER_URL, ALIAS_NAME, CERT_NAME and run the script: As a result, an trusted jks storage will be created with […]

READ MORE

Spark’s User Defined Functions in Java

In this article we will find the answer for questions: How to change the column in Spark? How to modify column in Spark? In other words: how to create a user defined function (UDF) and apply it. For example, let’s have a look to UDF, that takes a String and returns a String. For Spark […]

READ MORE

Spark failed to connect to the MetaStore Server

The problem You may encounter errors like this when running a Spark script / application: Solutions If you do not need the MetaStore server, there are two ways to disable it. Please note that Spark version >= 2.x is required. The first way (via spark2-submit parameters) The second way (via SparkConf object) Java example: Scala […]

READ MORE

Spark custom parquet OutputCommiter

If you need to make your own implementation of OutputCommiter for spark parquet-output tasks, then first of all you need to make a class that extends from org.apache.hadoop.mapreduce.OutputCommiter: Further, regardless of implementation of the OutputCommiter, you need to register the full class name in the Hadoop (!) configuration of Spark like this: Example for Scala: […]

READ MORE

Spark concurrent write to same HDFS path

The problem Sometimes you need to run such a scenario when several Spark tasks write data along the same path to HDFS. During the execution of tasks, you may encounter some errors: Suppose we have one Spark task, that writes to the hdfs://data/test directory. At runtime, Spark will make a temporary directory: hdfs://data/test/_temporary/0. There is […]

READ MORE

Spring TCP server

Let’s suppose that you need configure a TCP server in Spring / Spring Boot application. We will make it that the server prints the client’s request on the screen, then sends a response to the client. First, let’s include the spring-integration-ip dependency. In Maven it will look like this: Like gRPC ideology, in this case […]

READ MORE

Terracotta Ehcache notes

Java has a caching standard – JCache. This standard is described in the JSR-107 specification. This standardized approach makes it easier to integrate with different tools that perform the same function. Thus, Spring Boot supports JCache integration. Therefore, whatever implementation of JCache we would use, the code remains the same. Some of the common JCache […]

READ MORE