Kea [LWN.net]

Kea

Posted May 30, 2025 13:11 UTC (Fri) by Sesse (subscriber, #53779)
In reply to: Kea by job
Parent article: Local vulnerabilities in Kea DHCP

It's also a bit sad how many things are gone from ISC dhcpd; deprecating dhcpd before Kea has full feature parity leaves some users in a bit of a bind.

I agree that Kea is overly complex; it fixes an issue almost nobody has (performance in installation with 100k+ nodes) in exchange for a bazillion daemons and a codebase that takes ages to compile even on my 5950X.

to post comments

Kea

Posted May 30, 2025 14:35 UTC (Fri) by zdzichu (subscriber, #17118) [Link] (1 responses)

> it fixes an issue almost nobody has (performance in installation with 100k+ nodes)

That's why Meta ❤️ Kea.

> a bazillion daemons

Where?

CGroup: /system.slice/kea-dhcp4.service
└─3797 /usr/sbin/kea-dhcp4 -c /etc/kea/kea-dhcp4.conf

Kea

Posted May 30, 2025 14:40 UTC (Fri) by Sesse (subscriber, #53779) [Link]

dhcp4-server, dhcp6-server, dhcp-ddns, the control agent, and then some additional stuff for Stork (agent and server, IIRC) if you want to set up basic stats. And all of this needs to talk over sockets that have to be configured and by default are not very well secured.

Kea

Posted May 30, 2025 14:51 UTC (Fri) by DemiMarie (subscriber, #164188) [Link] (7 responses)

Also Kea is clearly latency-bound rather than throughput-bound when it comes to backend storage, which tells me there were design mistakes. A solution that is asynchronous and properly batches storage operations should be able to achieve much higher throughput.

Kea

Posted May 30, 2025 14:54 UTC (Fri) by Sesse (subscriber, #53779) [Link] (6 responses)

Is this with the CSV backend? (Or: What is it that you're doing with Kea that's putting so much demand on your storage?)

Kea’s missing transaction batching

Posted May 31, 2025 19:44 UTC (Sat) by DemiMarie (subscriber, #164188) [Link] (5 responses)

What I mean is that any database should be able to handle thousands of operations per second, provided that the client performs transactions in batches so that per-transaction overheads (such as network and fsync() latency) are amortized across the entire batch. A DHCP server could be designed to take advantage of this by submitting an entire batch of DHCP transactions to the database at once. While the database is performing its operation, accumulate the next batch of transactions. When the database operation finishes, submit the next batch and send acknowledgements for the first batch. Repeat as long as there are events.

The documentation at https://kb.isc.org/docs/kea-performance-optimization is clear that external lease file backends are expensive, so presumably there are users for which this is a problem. External lease file backend would not be a performance problem in a system that operates as I described above: they would only impact latency, not throughput.

Kea’s missing transaction batching

Posted May 31, 2025 20:07 UTC (Sat) by Sesse (subscriber, #53779) [Link] (4 responses)

OK, so you're basically asking that Kea buffer up changes and submit them all within one SQL transaction to the database backend? It sounds like something that is normally done in the database level (i.e., grouping fsyncs), but of course, that requires Kea to expose some parallelism as you say. (Either through async, or using more threads.) Anything else is fairly complex, as I believe Kea would want the lease commit to go through before it actually sends out the DHCPACK? (I haven't checked, I'm fine with memfile.)

Kea’s missing transaction batching

Posted Jun 1, 2025 4:37 UTC (Sun) by DemiMarie (subscriber, #164188) [Link] (3 responses)

You are correct that the commit should happen before sending DHCPACK, but that’s easy to ensure by buffering the DHCPACK packets until the whole batch of transactions has committed. In theory, databases should do the batching you describe, but I believe at least PostgreSQL can’t due to internal limitations.

Kea’s missing transaction batching

Posted Jun 1, 2025 7:32 UTC (Sun) by Sesse (subscriber, #53779) [Link] (2 responses)

In Postgres, commit_delay governs this behavior.

Kea’s missing transaction batching

Posted Jun 1, 2025 11:40 UTC (Sun) by andresfreund (subscriber, #69562) [Link] (1 responses)

And batching happens even without that setting, if multiple transactions start to flush while another flush is in progress.

PostgreSQL parallelism

Posted Jun 2, 2025 18:59 UTC (Mon) by DemiMarie (subscriber, #164188) [Link]

Does PostgreSQL support processing multiple transactions in parallel? Can it issue writes and reads with the high queue depths needed to get optimum performance out of modern storage?

Kea

Posted Jun 3, 2025 4:13 UTC (Tue) by raven667 (subscriber, #5198) [Link]

Yeah, dhcpd was battle tested and worked fine, although more complex configurations were ... complex, and I'm not sure it's bespoke config syntax is better than just using JSON like Kea. Unfortunately for me I _do_ have one of those sites with 100k+ clients, and while HA dhcpd worked OK in normal circumstances, if there is any network instability and clients have to retry it can become overwhelmed at that scale, peg 100% CPU on the single core it runs on, and your network doesn't work which makes for a lot of unhappy people. Kea seems to have about 10x higher limits on performance, the fact its not higher is probably related to locking since it's still managing one shared datastructure but I don't have enough CS background or time to figure it out exactly, but I'm sure the smarties at ISC did something reasonable.

I imagine the Enterprise distros that shipped dhcpd in their last release will continue to maintain it for the next 5+ years, and it's not as if there should be a ton of maintenance in such a mature codebase.