1570673066
In this article, I am going to make a basic example of how to connect to a “Redis Sentinel” configuration in a Spring application.
You might also enjoy: Getting Started with Python, Redis, and Nginx
First of all, for Maven dependencies, “spring-boot-starter-data-redis” is needed in our pom.xml file. This library includes LettuceConnectionFactory, which we will use to connect “Redis Sentinel”.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>2.0.5.RELEASE</version>
</dependency>
We will use application.yml file for connection and cache configurations. Below there is an example of application.yml file. Under cache.config.defaultConfig there is a list of predefined cache configurations. You can map these configurations to service cache definitions which are defined under cache.appCaches.appCacheMap.
cache:
config:
defaultConfigs:
- cacheName: ONE_HOUR_CACHE
timeToLiveSeconds: 3600
- cacheName: TEN_MINUTES_CACHE
timeToLiveSeconds: 600
appCaches:
appCacheMap:
serviceOne_cache: ONE_HOUR_CACHE
serviceTwo_cache: TEN_MINUTES_CACHE
spring:
cache:
type: redis
redis:
port: 6666
password: 123pwd
sentinel:
master: masterredis
nodes:
- 10.0.0.16
- 10.0.0.17
- 10.0.0.18
lettuce:
shutdown-timeout: 200ms
In order to read these configuration parameters easily, we will need a CacheSettingsModel and two classes annotated with ConfigurationProperties. One for the defaultConfigs and another for the appCacheMap. For redis configs, which are under spring.redis, we will use RedisProperties class, which is included in spring-boot-starter-data-redis. You can see these classes below:
package com.example.sentinel.config.model;
public class CacheSettingsModel {
private String cacheName;
private String timeToLiveSeconds;
public final String getCacheName() {
return cacheName;
}
public final void setCacheName(String cacheName) {
this.cacheName = cacheName;
}
public final String getTimeToLiveSeconds() {
return timeToLiveSeconds;
}
public final void setTimeToLiveSeconds(String timeToLiveSeconds) {
this.timeToLiveSeconds = timeToLiveSeconds;
}
}
package com.example.sentinel.config;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.apache.commons.collections4.CollectionUtils;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import com.example.sentinel.config.model.CacheSettingsModel;
@Component
@ConfigurationProperties( prefix="cache.config")
public class CacheSettings {
private List<CacheSettingsModel> defaultConfigs;
public List<CacheSettingsModel> getDefaultConfigs() {
return defaultConfigs;
}
public void setDefaultConfigs(List<CacheSettingsModel> defaultConfigs) {
this.defaultConfigs = defaultConfigs;
}
public Map<String,CacheSettingsModel> getCacheConfigAsMap() {
return CollectionUtils.emptyIfNull(defaultConfigs).stream().
collect(Collectors.toMap(CacheSettingsModel::getCacheName, c -> c));
}
}
package com.example.sentinel.config;
import java.util.Map;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
@Component
@ConfigurationProperties( prefix="cache.appCaches")
public class AppCacheSettings {
private Map<String,String> appCacheMap;
public Map<String, String> getAppCacheMap() {
return appCacheMap;
}
public void setAppCacheMap(Map<String, String> appCacheMap) {
this.appCacheMap = appCacheMap;
}
}
Now we are going to create a connection to “Redis Sentinel” with the help of those configuration classes. As I told you before, we are going to use Lettuce to connect to Redis. We are going to create a new instance of RedisSentinelConfiguration class with the help of RedisProperties class and use it in the constructor of LettuceConnectionFactory.
package com.example.sentinel.config;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import javax.inject.Inject;
import org.springframework.boot.autoconfigure.data.redis.RedisProperties;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisPassword;
import org.springframework.data.redis.connection.RedisSentinelConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import com.example.sentinel.config.AppCacheSettings;
import com.example.sentinel.config.CacheSettings;
import com.example.sentinel.config.model.CacheSettingsModel;
@Configuration
@EnableCaching
public class CacheConfig {
@Inject
private CacheSettings cacheSettings;
@Inject
private AppCacheSettings appCacheSettings;
@Inject
private RedisProperties redisProperties;
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
.master(redisProperties.getSentinel().getMaster());
redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, Integer.valueOf(redisProperties.getPort())));
sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
return new LettuceConnectionFactory(sentinelConfig);
}
@Bean
public RedisCacheConfiguration cacheConfiguration() {
return RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(600))
.disableCachingNullValues();
}
private RedisCacheConfiguration buildRedisCacheConfig(CacheSettingsModel cachesProperties) {
return RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(Long.parseLong(cachesProperties.getTimeToLiveSeconds())))
.disableCachingNullValues();
}
@Bean
public RedisCacheManager cacheManager() {
Map<String, RedisCacheConfiguration> cacheConfigs = new HashMap<>();
Map<String, CacheSettingsModel> cacheConfigMap = cacheSettings.getCacheConfigAsMap();
Map<String, String> appCacheMap = appCacheSettings.getAppCacheMap();
appCacheMap.forEach((key, value) -> cacheConfigs.put(key, buildRedisCacheConfig(cacheConfigMap.get(value))));
return RedisCacheManager.builder(redisConnectionFactory())
.cacheDefaults(cacheConfiguration())
.withInitialCacheConfigurations(cacheConfigs)
.transactionAware()
.build();
}
}
At the end, we are going to create a CacheManager bean. We are going to use RedisCacheManager.builder. With the help of AppCacheSettings and CacheSettings classes we are going to set initial cache configurations of cacheManager bean. If you want to enable transactional operations on caches you should set transactionAware to true (default is false). I’ll just copy the definition of transactionAware from the Spring API document.
Set this to “true” to synchronize cache put/evict operations with ongoing Spring-managed transactions, performing the actual cache put/evict operation only in the after-commit phase of a successful transaction.
We are almost done. To use Redis cache in our application, we only need to put @Cacheable (org.springframework.cache.annotation.Cacheable) annotation to the services that we want to cache. As a Value parameter, you can use one of the predefined cache names under appCacheMap properties (in application.yml file) to set cache properties like timeToLive.
@Cacheable("serviceOne_cache")
public List<String> serviceOne(String param) {
...
}
That’s it. Spring will do the rest!
Thank for visiting and reading this post.! We highly appreciate your actions! Please share if you liked it!
Database Caching With Redis and Java
#spring-boot #redis #database #web-development
1596679140
Redis offers two mechanisms for handling transactions – MULTI/EXEC based transactions and Lua scripts evaluation. Redis Lua scripting is the recommended approach and is fairly popular in usage.
Our Redis™ customers who have Lua scripts deployed often report this error – “BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE”. In this post, we will explain the Redis transactional property of scripts, what this error is about, and why we must be extra careful about it on Sentinel-managed systems that can failover.
Redis “transactions” aren’t really transactions as understood conventionally – in case of errors, there is no rollback of writes made by the script.
“Atomicity” of Redis scripts is guaranteed in the following manner:
It is highly recommended that the script complete within a time limit. Redis enforces this in a weak manner with the ‘lua-time-limit’ value. This is the maximum allowed time (in ms) that the script is allowed to run. The default value is 5 seconds. This is a really long time for CPU-bound activity (scripts have limited access and can’t run commands that access the disk).
However, the script is not killed when it executes beyond this time. Redis starts accepting client commands again, but responds to them with a BUSY error.
If you must kill the script at this point, there are two options available:
It is usually better to just wait for the script to complete its operation. The complete information on methods to kill the script execution and related behavior are available in the documentation.
#cloud #database #developer #high availability #howto #redis #scalegrid #lua-time-limit #redis diagram #redis master #redis scripts #redis sentinel #redis servers #redis transactions #sentinel-managed #server failures
1597222800
In our previous posts in this series, we spoke at length about using PgBouncer and Pgpool-II , the connection pool architecture and pros and cons of leveraging one for your PostgreSQL deployment. In our final post, we will put them head-to-head in a detailed feature comparison and compare the results of PgBouncer vs. Pgpool-II performance for your PostgreSQL hosting !
The bottom line – Pgpool-II is a great tool if you need load-balancing and high availability. Connection pooling is almost a bonus you get alongside. PgBouncer does only one thing, but does it really well. If the objective is to limit the number of connections and reduce resource consumption, PgBouncer wins hands down.
It is also perfectly fine to use both PgBouncer and Pgpool-II in a chain – you can have a PgBouncer to provide connection pooling, which talks to a Pgpool-II instance that provides high availability and load balancing. This gives you the best of both worlds!
PostgreSQL Connection Pooling: Part 4 – PgBouncer vs. Pgpool-II
While PgBouncer may seem to be the better option in theory, theory can often be misleading. So, we pitted the two connection poolers head-to-head, using the standard pgbench tool, to see which one provides better transactions per second throughput through a benchmark test. For good measure, we ran the same tests without a connection pooler too.
All of the PostgreSQL benchmark tests were run under the following conditions:
We ran each iteration for 5 minutes to ensure any noise averaged out. Here is how the middleware was installed:
Here are the transactions per second (TPS) results for each scenario across a range of number of clients:
#database #developer #performance #postgresql #connection control #connection pooler #connection pooler performance #connection queue #high availability #load balancing #number of connections #performance testing #pgbench #pgbouncer #pgbouncer and pgpool-ii #pgbouncer vs pgpool #pgpool-ii #pooling modes #postgresql connection pooling #postgresql limits #resource consumption #throughput benchmark #transactions per second #without pooling
1623501300
This article talks about master-slave configuration set up in local docker setup and we will verify read replica of slave nodes when master is unavailable or down. We can read data of not when the master node is down. This sample is explained with Spring Boot.
In realtime, the application has to process and fetch from multiple tables through joining to get complex data every time. To improve performance, if we could cache the response which does not change frequently, so that performance will be faster.
When cache is implemented there will be a 100% probability that a single instance can be hit with maximum load on a single instance. For Ex: an app service or back-end system continuously sends requests to a single instance then there might be chances where the connections in Redis instance may be exhausted.
To solve this, We can run multiple instances of Redis with master-slave architecture, where the master would be the write node and slaves would act as read-only nodes like scaling horizontally. Any updates to the master would be automatically synced with slave nodes asynchronously. Any write attempt to the slave nodes would be rejected.
#architecure #master-slave #redis #spring-boot-2 #redis master slave configuration and tested in spring boot #redis master slave configuration
1620720872
As an extension of the Spring Framework, Spring Boot is widely used to make development on Spring faster, more efficient and convenient. In this article, we will look at some of the parameters were using Spring Boot can drastically reduce the time and effort required in application development.
#full stack development #spring #spring and spring boot #spring boot
1623559620
Spring Native, for compiling Spring Java applications to standalone executables called native images, is now available as a beta release. Native images promise faster startup times and lower runtime memory overhead compared to the JVM.
Launched March 11 and available on start.spring.io, the Spring Native beta compiles Spring applications to native images using the GraalVM multi-language runtime. These standalone executables offer benefits including nearly instant startup (typically fewer than 100ms), instant peak performance, and lower memory consumption, at the cost of longer build times and fewer runtime optimizations than the JVM.
#spring native turns spring apps into native executables #spring native #spring #native executables #spring apps