Redis 21

Redis 21

Recently I am learning Redis related knowledge, and I read Ali's redis development specification and Redis development and operation and maintenance book. It is divided into four directions: usage specifications, pitted commands, actual project operations, and operation and maintenance configuration. 21 points of attention when using Redis have been sorted out. I hope it will be helpful to everyone and learn together!

1. Redis usage specification

1.1, key specification points

When we design Redis keys, we should pay attention to the following points:

  • Prefix the business name with the key and separate it with a colon to prevent key conflicts from being overwritten. For example, live:rank:1
  • To ensure that the semantics of the key are clear, the length of the key should be less than 30 characters as much as possible.
  • The key must not contain special characters, such as spaces, line breaks, single and double quotes, and other escape characters.
  • Redis keys should be set to ttl as much as possible to ensure that unused keys can be cleaned up or eliminated in time.

1.2, the main points of value specification

The value of Redis cannot be set arbitrarily.

"The first point" , if a large number of bigKeys are stored, there will be problems, which will lead to slow queries, excessive memory growth, and so on.

  • If it is a String type, the size of a single value is controlled within 10k.
  • For hash, list, set, zset types, the number of elements generally does not exceed 5000.

"The second point" is to select a suitable data type. Many small partners only use Redis's String type, which comes up with set and get. In fact, Redis provides **"rich data structure types"**, some business scenarios are more suitable

hash, zset
Wait for other data results.

"Counter-example:"

set user: 666 :name jay User SET: 666 : Age  18 is duplicated code

"Positive Law"

User hmset: 666  name Jay Age  18 is   duplicated code

1.3. Set the expiration time for the Key, and pay attention to the keys of different businesses, and try to spread the expiration time a little

  • Because Redis data is stored in memory, and memory resources are very precious.
  • We generally use Redis as a cache, and **"not a database"**, so the life cycle of the key should not be too long.
  • Therefore, your key is generally recommended to use **"expire to set the expiration time"**.

If a large number of keys expire at a certain point in time, Redis may be stuck at that point in time, and even **"cache avalanche"** may occur. Therefore, in general, the expiration time of keys for different businesses should be scattered. . Sometimes, for the same business, you can also add a random value to the time to spread the expiration time.

1.4. It is recommended to use batch operations to improve efficiency

When we write SQL every day, we all know that batch operations will be more efficient. Updating 50 items at a time is more efficient than looping 50 times and updating one item each time. In fact, Redis operation commands are also the same.

A command executed by the Redis client can be divided into 4 processes: 1. Send command -> 2. Command queue -> 3. Command execution -> 4. Return result. 1 and 4 are called RRT (command execution round trip time). Redis provides ** "batch operation commands, such as mget, mset", etc., which can effectively save RRT. However, most commands do not support batch operations, such as hgetall, and mhgetall does not exist. "Pipeline"** can solve this problem.

What is Pipeline? It can assemble a set of Redis commands, transmit them to Redis through an RTT, and then return the execution results of this set of Redis commands to the client in order.

Let's first take a look at the model that executed n commands without using Pipeline:

Using Pipeline to execute n commands, the whole process requires 1 RTT, the model is as follows:

2. The commands that Redis has pitfalls

2.1. Use with caution
O(n)
Complexity commands such as
hgetall
,
smember
,
lrange
Wait

Because Redis executes commands in a single thread. The time complexity of commands such as hgetall and smember is O(n). When n continues to increase, the Redis CPU will continue to soar and block the execution of other commands.

Commands such as hgetall, smember, lrange, etc. are not necessarily unavailable. You need to comprehensively evaluate the amount of data and clarify the value of n before making a decision. For example, hgetall, if there are more hash elements n, you can give priority to using ** hscan **.

2.2 Use Redis's monitor command with caution

The Redis Monitor command is used to print out the commands received by the Redis server in real time. If we want to know what command operations the client has done to the redis server, we can use the Monitor command to view it, but it is generally used for "debugging" . Don't use it in production! Because "monitor command may cause redis memory to continue to soar."**

The monitor model is Jiangzi. It will output all the commands executed on the Redis server. Generally speaking, the QPS of the Redis server is very high, that is, if the monitor command is executed, the Redis server is in the output buffer of the Monitor client. There will be a lot of "inventory", which will take up a lot of Redis memory.

2.3. The keys command cannot be used in the production environment

The Redis Keys command is used to find all keys that match a given pattern. If you want to see how many keys of a certain type are in Redis, many friends think of using the keys command, as follows:

keys key prefix* copy code

But redis

keys
It is traversed matching, and the complexity is
O(n)
, The more data in the database, the slower it is. We know that redis is single-threaded. If there is a lot of data, the keys instruction will cause the redis thread to block, and the online service will also pause. The service will not resume until the instruction is executed. Therefore, "generally in a production environment, do not use the keys command" . The official document also states:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using sets.

scan keys O(n) redis ; ** **

scan supports incremental iterative commands, and incremental iterative commands also have disadvantages: for example, using the SMEMBERS command can return all elements currently contained in the set key, but for incremental iterative commands such as SCAN, because in During the incremental iteration of the keys, the keys may be modified, so the incremental iteration command can only provide limited guarantees for the returned elements.

2.4 Prohibit the use of fluxhall and flushdb

  • The Flushall command is used to clear the data of the entire Redis server (delete all keys of all databases).
  • The Flushdb command is used to clear all keys in the current database.

These two commands are atomic and will not terminate execution. Once the execution starts, the execution will not fail.

2.5 Pay attention to the use of the del command

What command do you generally use to delete a key? Is it a direct del? If you delete a key, you can use the del command directly. But, have you thought about the time complexity of del? Let's discuss it by situation:

  • If you delete a String type key, the time complexity is
    O (1)
    , "You can directly del" .
  • If you delete a List/Hash/Set/ZSet type, its complexity is
    O(n)
    , n represents the number of elements.

Therefore, if you delete a List/Hash/Set/ZSet key, the more elements, the slower it will be. "When n is large, pay special attention" , it will block the main thread. So, if we don't use del, how should we delete it?

  • If it is a List type, you can execute
    lpop or rpop
    Until all elements are deleted.
  • If it is of Hash/Set/ZSet type, you can execute it first
    hscan/sscan/scan
    Query, then execute
    hdel/srem/zrem
    Delete each element in turn.

2.6 Avoid using SORT, SINTER and other highly complex commands.

Executing more complex commands will consume more CPU resources and block the main thread. So you have to avoid executing such as

SORT, SINTER, SINTERSTORE, ZUNIONSTORE, ZINTERSTORE
Wait for the aggregation command, it is generally recommended to put it on the client to execute.

3. Project actual combat and pit avoidance operation

3.1 Points to note when using distributed locks

Distributed lock is actually the realization of a lock that controls different processes of a distributed system to access shared resources together. Distributed locks are required for business scenarios such as placing orders with spikes and grabbing red envelopes. We often use Redis as a distributed lock, mainly with these points of attention:

3.1.1 The two commands SETNX + EXPIRE are written separately (a typical error implementation example)

if (jedis.setnx(key_resource_id,lock_value) ==  1 ){  //lock     expire (key_resource_id, 100 );  //set expiration time     try {         do something   //Business request     }catch(){   }   finally {        jedis.del(key_resource_id);  //Release the lock     } } Copy code

If it is done

setnx
Lock, when the expire time is about to be executed, the process crashes or needs to be restarted for maintenance, then the lock will be "immortal", **"other threads will never get the lock"**, so it is generally distributed Locks cannot be implemented in this way.

3.1.2 SETNX + value value is the expiration time (some small partners implement this way, there are pits)

long expires = System.currentTimeMillis() + expireTime;  //System time + set expiration time String expiresStr = String.valueOf(expires); //If the current lock does not exist, return the lock successfully if  (jedis.setnx(key_resource_id, expiresStr) ==  1 ) {          return  true ; }  //If the lock already exists, get the expiration time of the lock String currentValueStr = jedis.get(key_resource_id); //  if (currentValueStr != null && Long.parseLong(currentValueStr) < System.currentTimeMillis()) {      //  redis getSet     String oldValueStr = jedis.getSet(key_resource_id, expiresStr);          if (oldValueStr != null && oldValueStr.equals(currentValueStr)) {          //           return true;     } }          // return false; }

** **

  • /
  • jedis.getSet()

3.1.3 SET SET EX PX NX

if jedis.set(key_resource_id, lock_value, "NX""EX"100s) == 1//     try {         do something  //     }catch(){   }   finally {        jedis.del(key_resource_id); //     } }

3.1.4 SET EX PX NX + ,

if jedis.set(key_resource_id, uni_request_id, "NX""EX"100s) == 1//     try {         do something  //     }catch(){   }   finally {        //Judge whether it is the lock added by the current thread, and release it        if  (uni_request_id.equals(jedis.get(key_resource_id))) {         jedis.del(lockKey);  //Release the lock         }     } } Copy code

Here, judging whether the lock added by the current thread and releasing the lock is not an atomic operation. If you call jedis.del() to release the lock, the lock may no longer belong to the current client, and the lock added by others will be released.

Generally use lua script instead. The lua script is as follows:

if  redis.call( 'get' ,KEYS[ 1 ]) == ARGV[ 1 ] then      return  redis.call( 'del' ,KEYS[ 1 ])  else    return  0 end; Copy code

3.1.5 Redisson framework + Redlock algorithm solves the problem of lock expired release, business incomplete execution + stand-alone problem

Redisson uses a

Watch dog
Solve the problem of lock expired release and the business is not executed. The schematic diagram of Redisson is as follows:

The above distributed locks still have stand-alone problems:

If thread one gets the lock on the Redis master node, but the locked key has not been synchronized to the slave node. At exactly this moment, when the master node fails, a slave node will be upgraded to a master node. Thread two can acquire the lock of the same key, but thread one has already acquired the lock, and the security of the lock is lost.

For stand-alone problems, the Redlock algorithm can be used. Friends who are interested can read my article, seven solutions! Discuss the correct use posture of Redis distributed lock

3.2 Cautions for Cache Coherency

  • If it is a read request, read the cache first, then read the database
  • If a write request, update the database first, and then write to the cache
  • Every time you update the data, you need to clear the cache
  • Caching generally needs to set a certain expiration invalidation
  • If the consistency requirements are high, biglog+MQ can be used to guarantee.

Friends who are interested, can read my article: In a concurrent environment, should we operate the database first or the cache first?

3.3 Properly evaluate the Redis capacity to avoid the invalidation of the previously set expiration time due to frequent set coverage.

We know that all the data structure types of Redis can set the expiration time. Suppose a string has an expiration time set, and if you reset it, the previous expiration time will be invalid.

Redis 

setKey
The source code is as follows:

void setKey(redisDb *db,robj *key,robj *val) {      if (lookupKeyWrite(db,key)==NULL) {        dbAdd(db,key,val);     } else {     dbOverwrite(db,key,val);     }     incrRefCount(val);     removeExpire(db,key);  //Remove expiration time     signalModifiedKey(db,key); } Copy code

In actual business development, at the same time, we must reasonably evaluate the capacity of Redis to avoid frequent set coverage, which will cause the key with the expiration time to become invalid. Novice Xiaobai is easy to make this mistake.

3.4 Cache penetration problem

Let's first look at a common way of using the cache: when a read request comes, check the cache first, and if the cache has a value hit, it will return directly; if the cache misses, it will check the database, then update the value of the database to the cache, and then return.

  • userid
  • //

 

  1. API

0 N key N hash N N 1 1 key

3.5

  down

  • + 5 +0 1800
  • Redis Redis

3.6

  key Key db

down DB key key

  • 1. db (Redis setnx db
  • 2.

3.7 key

Redis key key key

Key

  • Redi key Hash Key

key

  • Key

key

  • Redis
  • key hash key key1,key2 keyN N N N
  • JVM , Redis

4. Redis

4.1

  • TCP redis redis
  • Redis

4.2 db0

Redis-standalone db0.

  • Redis select 0 select 1
  • Redis Cluster db0

4.3 maxmemory +

redis key redis maxmemory-policy( ) 8

  • volatile-lru key LRU
  • allkeys-lru key LRU
  • volatile-lfu 4.0 key LFU key
  • allkeys-lfu 4.0 key LFU
  • volatile-random key
  • allkeys-random key
  • volatile-ttl key
  • noeviction

4.4 lazy-free

Redis4.0+ lazy-free Redis bigKey lazy-free Redis bigkey

  • Redis KEYS [1]
  • Redis [2]
  • Redis 7 +43
  • Redis BloomFilter[3]
  • Redis [4]

Reference

[1]

Redis KEYS :www.cnblogs.com/tonyY/p/121

[2]

Redis :developer.aliyun.com/article/531

[3]

Redis BloomFilter:blog.csdn.net/wx152815940

[4]

Redis :www.shangmayuan.com/a/d2f178b54

Python \

5 Python \

5 \

/

\