Perhaps slow devices such as USB sticks should not get writeback caching. Writes to them should happen synchronously - that would also avoid the user-unfriendly extra step of unmounting the stick before unplugging it. If you ask to copy a file to a USB stick, the OS shouldn't lie when reporting that the operation has completed; it should only say it's done when it's done.
Or at least, have at most a fixed small amount of dirty pages for removable and slow devices (those that can be written within one second, say).