|
|
Subscribe / Log in / New account

One billion files on Linux

One billion files on Linux

Posted Aug 19, 2010 15:20 UTC (Thu) by zzxtty (guest, #45175)
Parent article: One billion files on Linux

If people are wondering about the validity of 1 billion files I can give an example, I work with MRI data. We do a lot of fMRI which generates lots of files (DICOM images). One file is generated per slice, with fMRI you continuously scan someone for an extended period of time, a single scan can generated 20,000 files. If you've got several MRI scanners and have been up and running for a few years the 1 billion file mark is not large, So far this year we have generated over 23 million files on one of our scanners.

However, I'm not sure I'd want to store them all on one file system, it would be a nightmare to restore from tape if anything did go horribly wrong. This is where data management comes in, I create a new partition for each scanner, each year. Currently we run all this on midrange hardware raid and format with zfs, it appears to cope. Would be nice to move it all to Linux =)


to post comments


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds