Insane number of files created per second

Insane number of files created per second

Posted Aug 27, 2016 12:07 UTC (Sat) by walex (guest, #69836)
In reply to: Insane number of files created per second by josefbacik
Parent article: Btrfs and high-speed devices

«we don't personally _need_ 200k files/sec, the workload quickly shows us where we have pain points that would cause us problems with real world workloads.»

That is about absolute speed of metadata operations, and that's not the real issue. The real issue being described here is that metadata operations don't scale with hardware capacity, regardless of the absolute speed desires, that is the real issue is about design.

The real issue exists because it is relatively easy to have scalable data speeds: just choose a domain which is "easily parallelizable" and throw more disks more RAM more threads at it. For data, RAID scales up speed pretty nicely.

By contrast metadata operations are not easily parallelizable, because there are dependencies across metadata, both structural dependencies and ordering dependencies, and therefore fairly fine grained locking must be used (ordering) and RAID does not work as well (structural).

The biggest problem with hard-to-parallelize metadata is not even file creation rates, it is whole-tree scans, like fsck or RSYNC scans. I have seen a lot of cases where some "clever" person designed a storage subsystem for average data workloads, and that became catastrophes during peak metadata workloads, which must happen quite periodically, one way or another.

---

to post comments