The influence of time_wait in development in TCP connection

The influence of time_wait in development in TCP connection

  According to the 3-way handshake disconnection regulation defined by the TCP protocol, the socket of the party that initiates the active closing of the socket will enter the TIME_WAIT state, the TIME_WAIT state will continue for 2 MSL (Max Segment Lifetime), and the socket in the TIME_WAIT state cannot be recycled. Specific phenomena For a server that handles a large number of short connections, if the server actively closes the client connection, it will cause a large number of sockets in the TIME_WAIT state on the server side, even more than the sockets in the Established state, which seriously affects the processing of the server. Ability, even exhaust the available sockets, stop the service. TIME_WAIT is a mechanism used by the TCP protocol to ensure that the re-allocated sockets will not be affected by the remaining delayed retransmission messages. It is a necessary logical guarantee.

netstat -an|awk'/tcp/{print $6}'|sort|uniq -c
     16 CLOSING
    298 FIN_WAIT1
     13 FIN_WAIT2
      9 LAST_ACK
      7 LISTEN
    103 SYN_RECV
   5204 TIME_WAIT
status description
CLOSED: No connection is active or ongoing
LISTEN: The server is waiting for an incoming call
SYN_RECV: A connection request has arrived, waiting for confirmation
SYN_SENT: The application has started, open a connection
ESTABLISHED: Normal data transmission status
FIN_WAIT1: The application says it is finished
FIN_WAIT2: The other side has agreed to release
ITMED_WAIT: Wait for all packets to die
CLOSING: Both sides try to close at the same time
TIME_WAIT: A release has been initialized on the other side
LAST_ACK: Wait for all packets to die
netstat -ae |grep mysql
tcp 0 0 aaaa:53045 TIME_WAIT root 0
tcp 0 0 aaaa:53044 TIME_WAIT root 0
tcp 0 0 aaaa:53051 TIME_WAIT root 0
tcp 0 0 aaaa:53050 TIME_WAIT root 0
tcp 0 0 aaaa:53049 TIME_WAIT root 0

It is found that there are a large number of connections in the TIME_WAIT state in the system, which can be solved by adjusting the kernel parameters.


Edit the file and add the following content:

net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200    
net.ipv4.ip_local_port_range = 1024 65000 ## Port allocation range
net.ipv4.tcp_max_tw_buckets = 5000 ## Set "time_wait" buckets up to 5000
Then execute/sbin/sysctl -p to make the parameters take effect.
net.ipv4.tcp_syncookies = 1 means to enable SYN Cookies. When the SYN waiting queue overflows, cookies are enabled for processing, which can prevent a small amount of SYN attacks. The default is 0, which means it is closed;
net.ipv4.tcp_tw_reuse = 1 means to enable reuse. Allow TIME-WAIT sockets to be reused for new TCP connections, the default is 0, which means it is closed;
net.ipv4.tcp_tw_recycle = 1 means to turn on the fast recycling of TIME-WAIT sockets in the TCP connection. The default is 0, which means it is turned off.
net.ipv4.tcp_fin_timeout = 30 means that if the socket is closed by the local request, this parameter determines the time it will keep in the FIN-WAIT-2 state.
net.ipv4.tcp_keepalive_time = 1200 indicates how often TCP sends keepalive messages when keepalive is enabled. The default is 2 hours, change to 20 minutes.
net.ipv4.ip_local_port_range = 1024 65000 indicates the port range used for external connections. The default is very small: 32768 to 61000, changed to 1024 to 65000.
net.ipv4.tcp_max_syn_backlog = 8192 represents the length of the SYN queue, the default is 1024, increasing the queue length to 8192, can accommodate more network connections waiting to be connected.
net.ipv4.tcp_max_tw_buckets = 5000 means that the system maintains the maximum number of TIME_WAIT sockets at the same time. If this number is exceeded, the TIME_WAIT socket will be cleared immediately and a warning message will be printed. The default is 180000, changed to 5000.

[Recommended by Alibaba java development manual] Highly concurrent server is recommended to reduce the time_wait timeout time of TCP protocol.

Note: By default, the operating system will close the connection in the time_wait state after 240 seconds. Under high concurrent access, the server may not be able to establish a new connection because there are too many connections in time_wait, so it needs to be reduced on the server. Wait for the value.

Positive example: Please modify the default value (seconds) by changing the/etc/sysctl.conf file on the linux server:

net.ipv4.tcp_fin_timeout = 30. 

Searching for people to fish is not as good as teaching them to fish. Let’s review part of the detailed explanation of TCP protocol in computer network, excerpted from-Xie Xiren Computer Network

The three stages of TCP's transport connection There are three stages in the transport connection, namely: connection establishment, data transmission and connection release. The management of transportation connections is to enable the establishment and release of transportation connections to proceed normally. The following three problems must be solved during the connection establishment process: Each party must be able to know the other's existence. To allow both parties to negotiate some parameters (such as the maximum segment length, maximum window size, quality of service, etc.). Be able to allocate transport entity resources (such as cache size, items in the connection table, etc.). The client-server TCP connection is established in the client-server method. The application process that actively initiates connection establishment is called a client. The application process that passively waits for a connection to be established is called a server. TCP connection establishment

A's TCP sends a connection request segment to B, with the synchronization bit SYN = 1 in the header, and selects the sequence number seq = x, indicating that the sequence number of the first data byte when transmitting data is x. After B's TCP receives the connection request segment, if it agrees, it sends back an acknowledgment. B should make SYN = 1 in the acknowledgment segment, ACK = 1, its acknowledgment number ack = x + 1, and the sequence number seq = y of its own choice.

After receiving this segment, A gives an acknowledgment to B, its ACK = 1, and the acknowledgment number ack = y + 1. A's TCP informs the upper application process that the connection has been established.

After receiving the confirmation from host A, TCP of B also informs its upper application process that the TCP connection has been established.

Use the three-way handshake to establish each state of the TCP connection

TCP connection release

After the data transmission is over, both parties in the communication can release the connection . Now A's application process first sends a connection release segment to its TCP, stops sending data again, and actively closes the TCP connection. A puts FIN = 1 in the header of the connection release message segment, its sequence number seq = u, and waits for B's confirmation.

B sends an acknowledgment, the acknowledgment number ack = u + 1, and the segment's own sequence number seq = v.

The TCP server process notifies the higher-level application process. The connection from A to B in this direction is released, and the TCP connection is half-closed. If B sends data, A still has to receive it. If B has no data to send to A, its application process will notify TCP to release the connection. After A receives the connection release segment, it must send an acknowledgment. In the confirmation segment, ACK = 1, confirmation number ack = w + 1, own serial number seq = u + 1.

The TCP connection must be released after the time 2MSL has passed.

1. to ensure that the last ACK segment sent by A can reach B. 2. prevent "failed connection request segment" from appearing in this connection. A After sending the last ACK message segment, and then after 2MSL, all the message segments generated during the duration of this connection can disappear from the network. In this way, the old connection request message segment will not appear in the next new connection.