| From: |
| Eric Dumazet <edumazet-AT-google.com> |
| To: |
| "David S . Miller" <davem-AT-davemloft.net>, Jakub Kicinski <kuba-AT-kernel.org>, Paolo Abeni <pabeni-AT-redhat.com> |
| Subject: |
| [PATCH v3 net-next 00/14] net_sched: speedup qdisc dequeue |
| Date: |
| Fri, 21 Nov 2025 08:32:42 +0000 |
| Message-ID: |
| <20251121083256.674562-1-edumazet@google.com> |
| Cc: |
| Simon Horman <horms-AT-kernel.org>, Jamal Hadi Salim <jhs-AT-mojatatu.com>, Cong Wang <xiyou.wangcong-AT-gmail.com>, Jiri Pirko <jiri-AT-resnulli.us>, "Toke Høiland-Jørgensen" <toke-AT-redhat.com>, Kuniyuki Iwashima <kuniyu-AT-google.com>, Willem de Bruijn <willemb-AT-google.com>, netdev-AT-vger.kernel.org, eric.dumazet-AT-gmail.com, Eric Dumazet <edumazet-AT-google.com> |
| Archive-link: |
| Article |
Avoid up to two cache line misses in qdisc dequeue() to fetch
skb_shinfo(skb)->gso_segs/gso_size while qdisc spinlock is held.
Idea is to cache gso_segs at enqueue time before spinlock is
acquired, in the first skb cache line, where we already
have qdisc_skb_cb(skb)->pkt_len.
This series gives a 8 % improvement in a TX intensive workload.
(120 Mpps -> 130 Mpps on a Turin host, IDPF with 32 TX queues)
v3: - Same than v2, resent after a syzbot report was fixed in commit
4ef927436258 ("bpf: Add bpf_prog_run_data_pointers()")
v2: - Fixed issues reported by Jakub (thanks !)
- Added three patches adding/using qdisc_dequeue_drop() after
recent regressions with CAKE qdisc reported by Toke.
More fixes to come later.
- https://lore.kernel.org/netdev/20251111093204.1432437-1-e...
v1: https://lore.kernel.org/netdev/20251110094505.3335073-1-e...
Eric Dumazet (14):
net_sched: make room for (struct qdisc_skb_cb)->pkt_segs
net: init shinfo->gso_segs from qdisc_pkt_len_init()
net_sched: initialize qdisc_skb_cb(skb)->pkt_segs in
qdisc_pkt_len_init()
net: use qdisc_pkt_len_segs_init() in sch_handle_ingress()
net_sched: use qdisc_skb_cb(skb)->pkt_segs in bstats_update()
net_sched: cake: use qdisc_pkt_segs()
net_sched: add Qdisc_read_mostly and Qdisc_write groups
net_sched: sch_fq: move qdisc_bstats_update() to fq_dequeue_skb()
net_sched: sch_fq: prefetch one skb ahead in dequeue()
net: prefech skb->priority in __dev_xmit_skb()
net: annotate a data-race in __dev_xmit_skb()
net_sched: add tcf_kfree_skb_list() helper
net_sched: add qdisc_dequeue_drop() helper
net_sched: use qdisc_dequeue_drop() in cake, codel, fq_codel
include/net/pkt_sched.h | 5 +-
include/net/sch_generic.h | 101 ++++++++++++++++++++++++++++----------
net/core/dev.c | 62 +++++++++++++----------
net/sched/act_ct.c | 8 +--
net/sched/cls_api.c | 6 +--
net/sched/cls_flower.c | 2 +-
net/sched/sch_cake.c | 19 +++----
net/sched/sch_codel.c | 4 +-
net/sched/sch_dualpi2.c | 1 +
net/sched/sch_fq.c | 9 ++--
net/sched/sch_fq_codel.c | 5 +-
net/sched/sch_netem.c | 1 +
net/sched/sch_qfq.c | 2 +-
net/sched/sch_taprio.c | 1 +
net/sched/sch_tbf.c | 1 +
15 files changed, 147 insertions(+), 80 deletions(-)
--
2.52.0.460.gd25c4c69ec-goog
