The Linux "copy problem"

Posted Jun 1, 2019 16:11 UTC (Sat) by jmclnx (guest, #72456)
Parent article: The Linux "copy problem"

Speeding up things is always good, but the one thing I have noticed in the last few years. Once in a great while, if I cp a large file from one local fs to another local fs, the whole system would freeze for multiple 10s of seconds when a sync happens.

In the early days I never saw this and we use to make fun of Microsoft for having that freeze issue. I would rather have a slower cp and not have that sync freeze :)

to post comments

The Linux "copy problem"

Posted Jun 1, 2019 21:13 UTC (Sat) by desbma (guest, #118820) [Link]

This is most likely the infamous "dirty writeback" problem, best described in this LWN article: https://lwn.net/Articles/572911/

On my systems, I fix it permanently with:
echo "vm.dirty_background_bytes=$((48 * 1024 * 1024))
vm.dirty_ratio=10" | sudo tee /etc/sysctl.d/99-dirty-writeback.conf
sudo sysctl --system

That way the writeback kicks in with at most 48MB of dirty data, and also stalls the writer process before there is more than 10% of all memory consumed by dirty writeback pages.