1570673066
In this article, I am going to make a basic example of how to connect to a “Redis Sentinel” configuration in a Spring application.
You might also enjoy: Getting Started with Python, Redis, and Nginx
First of all, for Maven dependencies, “spring-boot-starter-data-redis” is needed in our pom.xml file. This library includes LettuceConnectionFactory, which we will use to connect “Redis Sentinel”.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>2.0.5.RELEASE</version>
</dependency>
We will use application.yml file for connection and cache configurations. Below there is an example of application.yml file. Under cache.config.defaultConfig there is a list of predefined cache configurations. You can map these configurations to service cache definitions which are defined under cache.appCaches.appCacheMap.
cache:
config:
defaultConfigs:
- cacheName: ONE_HOUR_CACHE
timeToLiveSeconds: 3600
- cacheName: TEN_MINUTES_CACHE
timeToLiveSeconds: 600
appCaches:
appCacheMap:
serviceOne_cache: ONE_HOUR_CACHE
serviceTwo_cache: TEN_MINUTES_CACHE
spring:
cache:
type: redis
redis:
port: 6666
password: 123pwd
sentinel:
master: masterredis
nodes:
- 10.0.0.16
- 10.0.0.17
- 10.0.0.18
lettuce:
shutdown-timeout: 200ms
In order to read these configuration parameters easily, we will need a CacheSettingsModel and two classes annotated with ConfigurationProperties. One for the defaultConfigs and another for the appCacheMap. For redis configs, which are under spring.redis, we will use RedisProperties class, which is included in spring-boot-starter-data-redis. You can see these classes below:
package com.example.sentinel.config.model;
public class CacheSettingsModel {
private String cacheName;
private String timeToLiveSeconds;
public final String getCacheName() {
return cacheName;
}
public final void setCacheName(String cacheName) {
this.cacheName = cacheName;
}
public final String getTimeToLiveSeconds() {
return timeToLiveSeconds;
}
public final void setTimeToLiveSeconds(String timeToLiveSeconds) {
this.timeToLiveSeconds = timeToLiveSeconds;
}
}
package com.example.sentinel.config;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.apache.commons.collections4.CollectionUtils;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import com.example.sentinel.config.model.CacheSettingsModel;
@Component
@ConfigurationProperties( prefix="cache.config")
public class CacheSettings {
private List<CacheSettingsModel> defaultConfigs;
public List<CacheSettingsModel> getDefaultConfigs() {
return defaultConfigs;
}
public void setDefaultConfigs(List<CacheSettingsModel> defaultConfigs) {
this.defaultConfigs = defaultConfigs;
}
public Map<String,CacheSettingsModel> getCacheConfigAsMap() {
return CollectionUtils.emptyIfNull(defaultConfigs).stream().
collect(Collectors.toMap(CacheSettingsModel::getCacheName, c -> c));
}
}
package com.example.sentinel.config;
import java.util.Map;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
@Component
@ConfigurationProperties( prefix="cache.appCaches")
public class AppCacheSettings {
private Map<String,String> appCacheMap;
public Map<String, String> getAppCacheMap() {
return appCacheMap;
}
public void setAppCacheMap(Map<String, String> appCacheMap) {
this.appCacheMap = appCacheMap;
}
}
Now we are going to create a connection to “Redis Sentinel” with the help of those configuration classes. As I told you before, we are going to use Lettuce to connect to Redis. We are going to create a new instance of RedisSentinelConfiguration class with the help of RedisProperties class and use it in the constructor of LettuceConnectionFactory.
package com.example.sentinel.config;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import javax.inject.Inject;
import org.springframework.boot.autoconfigure.data.redis.RedisProperties;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisPassword;
import org.springframework.data.redis.connection.RedisSentinelConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import com.example.sentinel.config.AppCacheSettings;
import com.example.sentinel.config.CacheSettings;
import com.example.sentinel.config.model.CacheSettingsModel;
@Configuration
@EnableCaching
public class CacheConfig {
@Inject
private CacheSettings cacheSettings;
@Inject
private AppCacheSettings appCacheSettings;
@Inject
private RedisProperties redisProperties;
@Bean
public LettuceConnectionFactory redisConnectionFactory() {
RedisSentinelConfiguration sentinelConfig = new RedisSentinelConfiguration()
.master(redisProperties.getSentinel().getMaster());
redisProperties.getSentinel().getNodes().forEach(s -> sentinelConfig.sentinel(s, Integer.valueOf(redisProperties.getPort())));
sentinelConfig.setPassword(RedisPassword.of(redisProperties.getPassword()));
return new LettuceConnectionFactory(sentinelConfig);
}
@Bean
public RedisCacheConfiguration cacheConfiguration() {
return RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(600))
.disableCachingNullValues();
}
private RedisCacheConfiguration buildRedisCacheConfig(CacheSettingsModel cachesProperties) {
return RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(Long.parseLong(cachesProperties.getTimeToLiveSeconds())))
.disableCachingNullValues();
}
@Bean
public RedisCacheManager cacheManager() {
Map<String, RedisCacheConfiguration> cacheConfigs = new HashMap<>();
Map<String, CacheSettingsModel> cacheConfigMap = cacheSettings.getCacheConfigAsMap();
Map<String, String> appCacheMap = appCacheSettings.getAppCacheMap();
appCacheMap.forEach((key, value) -> cacheConfigs.put(key, buildRedisCacheConfig(cacheConfigMap.get(value))));
return RedisCacheManager.builder(redisConnectionFactory())
.cacheDefaults(cacheConfiguration())
.withInitialCacheConfigurations(cacheConfigs)
.transactionAware()
.build();
}
}
At the end, we are going to create a CacheManager bean. We are going to use RedisCacheManager.builder. With the help of AppCacheSettings and CacheSettings classes we are going to set initial cache configurations of cacheManager bean. If you want to enable transactional operations on caches you should set transactionAware to true (default is false). I’ll just copy the definition of transactionAware from the Spring API document.
Set this to “true” to synchronize cache put/evict operations with ongoing Spring-managed transactions, performing the actual cache put/evict operation only in the after-commit phase of a successful transaction.
We are almost done. To use Redis cache in our application, we only need to put @Cacheable (org.springframework.cache.annotation.Cacheable) annotation to the services that we want to cache. As a Value parameter, you can use one of the predefined cache names under appCacheMap properties (in application.yml file) to set cache properties like timeToLive.
@Cacheable("serviceOne_cache")
public List<String> serviceOne(String param) {
...
}
That’s it. Spring will do the rest!
Thank for visiting and reading this post.! We highly appreciate your actions! Please share if you liked it!
Database Caching With Redis and Java
#spring-boot #redis #database #web-development
1596679140
Redis offers two mechanisms for handling transactions – MULTI/EXEC based transactions and Lua scripts evaluation. Redis Lua scripting is the recommended approach and is fairly popular in usage.
Our Redis™ customers who have Lua scripts deployed often report this error – “BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE”. In this post, we will explain the Redis transactional property of scripts, what this error is about, and why we must be extra careful about it on Sentinel-managed systems that can failover.
Redis “transactions” aren’t really transactions as understood conventionally – in case of errors, there is no rollback of writes made by the script.
“Atomicity” of Redis scripts is guaranteed in the following manner:
It is highly recommended that the script complete within a time limit. Redis enforces this in a weak manner with the ‘lua-time-limit’ value. This is the maximum allowed time (in ms) that the script is allowed to run. The default value is 5 seconds. This is a really long time for CPU-bound activity (scripts have limited access and can’t run commands that access the disk).
However, the script is not killed when it executes beyond this time. Redis starts accepting client commands again, but responds to them with a BUSY error.
If you must kill the script at this point, there are two options available:
It is usually better to just wait for the script to complete its operation. The complete information on methods to kill the script execution and related behavior are available in the documentation.
#cloud #database #developer #high availability #howto #redis #scalegrid #lua-time-limit #redis diagram #redis master #redis scripts #redis sentinel #redis servers #redis transactions #sentinel-managed #server failures
1597222800
In our previous posts in this series, we spoke at length about using PgBouncer and Pgpool-II , the connection pool architecture and pros and cons of leveraging one for your PostgreSQL deployment. In our final post, we will put them head-to-head in a detailed feature comparison and compare the results of PgBouncer vs. Pgpool-II performance for your PostgreSQL hosting !
The bottom line – Pgpool-II is a great tool if you need load-balancing and high availability. Connection pooling is almost a bonus you get alongside. PgBouncer does only one thing, but does it really well. If the objective is to limit the number of connections and reduce resource consumption, PgBouncer wins hands down.
It is also perfectly fine to use both PgBouncer and Pgpool-II in a chain – you can have a PgBouncer to provide connection pooling, which talks to a Pgpool-II instance that provides high availability and load balancing. This gives you the best of both worlds!
PostgreSQL Connection Pooling: Part 4 – PgBouncer vs. Pgpool-II
While PgBouncer may seem to be the better option in theory, theory can often be misleading. So, we pitted the two connection poolers head-to-head, using the standard pgbench tool, to see which one provides better transactions per second throughput through a benchmark test. For good measure, we ran the same tests without a connection pooler too.
All of the PostgreSQL benchmark tests were run under the following conditions:
We ran each iteration for 5 minutes to ensure any noise averaged out. Here is how the middleware was installed:
Here are the transactions per second (TPS) results for each scenario across a range of number of clients:
#database #developer #performance #postgresql #connection control #connection pooler #connection pooler performance #connection queue #high availability #load balancing #number of connections #performance testing #pgbench #pgbouncer #pgbouncer and pgpool-ii #pgbouncer vs pgpool #pgpool-ii #pooling modes #postgresql connection pooling #postgresql limits #resource consumption #throughput benchmark #transactions per second #without pooling
1641524460
The primary goal of the Spring Data project is to make it easier to build Spring-powered applications that use new data access technologies such as non-relational databases, map-reduce frameworks, and cloud based data services.
This modules provides integration with the Redis store.
RedisTemplate
that provides a high level abstraction for performing various Redis operations, exception translation and serialization support.Repository
interfaces including support for custom finder methods using @EnableRedisRepositories
.This project is governed by the Spring Code of Conduct. By participating, you are expected to uphold this code of conduct. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.
Here is a quick teaser of an application using Spring Data Redis in Java:
public class Example {
// inject the actual template
@Autowired
private RedisTemplate<String, String> template;
// inject the template as ListOperations
// can also inject as Value, Set, ZSet, and HashOperations
@Resource(name="redisTemplate")
private ListOperations<String, String> listOps;
public void addLink(String userId, URL url) {
listOps.leftPush(userId, url.toExternalForm());
// or use template directly
redisTemplate.boundListOps(userId).leftPush(url.toExternalForm());
}
}
@Configuration
class ApplicationConfig {
@Bean
public RedisConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory();
}
}
Add the Maven dependency:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<version>${version}</version>
</dependency>
If you’d rather like the latest snapshots of the upcoming major version, use our Maven snapshot repository and declare the appropriate dependency version.
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<version>${version}-SNAPSHOT</version>
</dependency>
<repository>
<id>spring-libs-snapshot</id>
<name>Spring Snapshot Repository</name>
<url>https://repo.spring.io/libs-snapshot</url>
</repository>
Having trouble with Spring Data? We’d love to help!
spring-data-redis
. You can also chat with the community on Gitter.Spring Data uses Github as issue tracking system to record bugs and feature requests. If you want to raise an issue, please follow the recommendations below:
You don’t need to build from source to use Spring Data (binaries in repo.spring.io), but if you want to try out the latest and greatest, Spring Data can be easily built with the maven wrapper. You also need JDK 1.8.
$ ./mvnw clean install
If you want to build with the regular mvn
command, you will need Maven v3.5.0 or above.
Also see CONTRIBUTING.adoc if you wish to submit pull requests, and in particular please sign the Contributor’s Agreement before your first non-trivial change.
Building the documentation builds also the project without running tests.
$ ./mvnw clean install -Pdistribute
The generated documentation is available from target/site/reference/html/index.html
.
The spring.io site contains several guides that show how to use Spring Data step-by-step:
Messaging with Redis: Learn how to use Redis as a message broker.
Accessing Data Reactively with Redis: Learn how to reactively interface with Redis and Spring Data.
Spring Data Examples contains example projects that explain specific features in more detail.
Download Details:
Author: spring-projects
Source Code: https://github.com/spring-projects/spring-data-redis
License: Apache-2.0 License
1623501300
This article talks about master-slave configuration set up in local docker setup and we will verify read replica of slave nodes when master is unavailable or down. We can read data of not when the master node is down. This sample is explained with Spring Boot.
In realtime, the application has to process and fetch from multiple tables through joining to get complex data every time. To improve performance, if we could cache the response which does not change frequently, so that performance will be faster.
When cache is implemented there will be a 100% probability that a single instance can be hit with maximum load on a single instance. For Ex: an app service or back-end system continuously sends requests to a single instance then there might be chances where the connections in Redis instance may be exhausted.
To solve this, We can run multiple instances of Redis with master-slave architecture, where the master would be the write node and slaves would act as read-only nodes like scaling horizontally. Any updates to the master would be automatically synced with slave nodes asynchronously. Any write attempt to the slave nodes would be rejected.
#architecure #master-slave #redis #spring-boot-2 #redis master slave configuration and tested in spring boot #redis master slave configuration
1620720872
As an extension of the Spring Framework, Spring Boot is widely used to make development on Spring faster, more efficient and convenient. In this article, we will look at some of the parameters were using Spring Boot can drastically reduce the time and effort required in application development.
#full stack development #spring #spring and spring boot #spring boot