| From: |
| Eric Dumazet <edumazet-AT-google.com> |
| To: |
| "David S . Miller" <davem-AT-davemloft.net>, Jakub Kicinski <kuba-AT-kernel.org>, Paolo Abeni <pabeni-AT-redhat.com>, Neal Cardwell <ncardwell-AT-google.com> |
| Subject: |
| [PATCH net-next 0/4] tcp: scale connect() under pressure |
| Date: |
| Sun, 02 Mar 2025 12:42:33 +0000 |
| Message-ID: |
| <20250302124237.3913746-1-edumazet@google.com> |
| Cc: |
| Kuniyuki Iwashima <kuniyu-AT-amazon.com>, Jason Xing <kerneljasonxing-AT-gmail.com>, Simon Horman <horms-AT-kernel.org>, netdev-AT-vger.kernel.org, eric.dumazet-AT-gmail.com, Eric Dumazet <edumazet-AT-google.com> |
| Archive-link: |
| Article |
Adoption of bhash2 in linux-6.1 made some operations almost twice
more expensive, because of additional locks.
This series adds RCU in __inet_hash_connect() to help the
case where many attempts need to be made before finding
an available 4-tuple.
This brings a ~200 % improvement in this experiment:
Server:
ulimit -n 40000; neper/tcp_crr -T 200 -F 30000 -6 --nolog
Client:
ulimit -n 40000; neper/tcp_crr -T 200 -F 30000 -6 --nolog -c -H server
Before series:
utime_start=0.288582
utime_end=1.548707
stime_start=20.637138
stime_end=2002.489845
num_transactions=484453
latency_min=0.156279245
latency_max=20.922042756
latency_mean=1.546521274
latency_stddev=3.936005194
num_samples=312537
throughput=47426.00
perf top on the client:
49.54% [kernel] [k] _raw_spin_lock
25.87% [kernel] [k] _raw_spin_lock_bh
5.97% [kernel] [k] queued_spin_lock_slowpath
5.67% [kernel] [k] __inet_hash_connect
3.53% [kernel] [k] __inet6_check_established
3.48% [kernel] [k] inet6_ehashfn
0.64% [kernel] [k] rcu_all_qs
After this series:
utime_start=0.271607
utime_end=3.847111
stime_start=18.407684
stime_end=1997.485557
num_transactions=1350742
latency_min=0.014131929
latency_max=17.895073144
latency_mean=0.505675853 # Nice reduction of latency metrics
latency_stddev=2.125164772
num_samples=307884
throughput=139866.80 # 194 % increase
perf top on client:
56.86% [kernel] [k] __inet6_check_established
17.96% [kernel] [k] __inet_hash_connect
13.88% [kernel] [k] inet6_ehashfn
2.52% [kernel] [k] rcu_all_qs
2.01% [kernel] [k] __cond_resched
0.41% [kernel] [k] _raw_spin_lock
Eric Dumazet (4):
tcp: use RCU in __inet{6}_check_established()
tcp: optimize inet_use_bhash2_on_bind()
tcp: add RCU management to inet_bind_bucket
tcp: use RCU lookup in __inet_hash_connect()
include/net/inet_hashtables.h | 7 ++--
net/ipv4/inet_connection_sock.c | 8 ++--
net/ipv4/inet_hashtables.c | 65 ++++++++++++++++++++++++---------
net/ipv4/inet_timewait_sock.c | 2 +-
net/ipv6/inet6_hashtables.c | 23 ++++++++++--
5 files changed, 75 insertions(+), 30 deletions(-)
--
2.48.1.711.g2feabab25a-goog
